-- Logs begin at Tue 2022-07-12 13:44:37 UTC, end at Tue 2022-07-12 14:58:37 UTC. -- Jul 12 14:19:32.918772 dv-cinder-820027-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 14:19:32.928400 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 14:19:32 2022] *** Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: nodename: dv-cinder-820027-12-iscsi Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: machine: x86_64 Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: clock source: unix Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: pcre jit disabled Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: detected number of CPU cores: 4 Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: current working directory: / Jul 12 14:19:32.929100 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: detected binary path: /usr/bin/uwsgi-core Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: your processes number limit is 47680 Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: your memory page size is 4096 bytes Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: detected max file descriptor number: 2048 Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: lock engine: pthread robust mutexes Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: thunder lock: enabled Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 14:19:32.930398 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 14:19:32.973475 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: Python main interpreter initialized at 0x55a7a27079c0 Jul 12 14:19:32.973475 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: python threads support enabled Jul 12 14:19:32.973475 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: your server socket listen backlog is limited to 100 connections Jul 12 14:19:32.973475 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: your mercy for graceful operations on workers is 90 seconds Jul 12 14:19:32.973829 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 14:19:32.973907 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: *** Operational MODE: preforking *** Jul 12 14:19:32.974154 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: *** uWSGI is running in multiple interpreter mode *** Jul 12 14:19:32.974154 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: spawned uWSGI master process (pid: 80334) Jul 12 14:19:32.974349 dv-cinder-820027-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 14:19:32.976042 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: spawned uWSGI worker 1 (pid: 80346, cores: 1) Jul 12 14:19:32.976042 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: spawned uWSGI worker 2 (pid: 80347, cores: 1) Jul 12 14:19:32.976042 dv-cinder-820027-12-iscsi devstack@keystone.service[80334]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 14:19:38.445177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:19:38.445177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: from cryptography.utils import int_from_bytes Jul 12 14:19:38.445840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:19:38.445840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: from cryptography.utils import int_from_bytes Jul 12 14:19:38.446288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:19:38.446288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: from cryptography.utils import int_from_bytes Jul 12 14:19:38.446422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:19:38.446422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: from cryptography.utils import int_from_bytes Jul 12 14:19:38.495201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 14:19:38.495373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 14:19:38.495460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 14:19:38.495549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 14:19:38.495631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] command line args: [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 14:19:38.495716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] command line args: [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 14:19:38.495853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 14:19:38.495985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 14:19:38.496641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ================================================================================ {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 14:19:38.496729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ================================================================================ {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 14:19:38.496803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] admin_token = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.496887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] admin_token = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] config_dir = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] config_dir = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] config_source = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] config_source = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] debug = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] debug = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.497832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.498936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_config_append = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_config_append = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_dir = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_dir = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_file = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.499856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_file = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.500898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.501928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.502968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.503955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] publish_errors = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] publish_errors = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.504892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] standard_threads = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] standard_threads = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.505936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] use_journal = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] use_journal = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] use_json = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] use_json = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] use_stderr = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] use_stderr = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] use_syslog = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] use_syslog = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.506976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:19:38.507113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.507933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.508908 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.509920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.510908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.511651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.512365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.512448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.512604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.512933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.513833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.514944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.515982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.516992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.517850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.518948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.519963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.520916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.521886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.522991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.523953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.524979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.525911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.526851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.527998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.528963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.529937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.530930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.531855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.532993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.external = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.external = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.533865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.password = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] auth.token = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.password = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] auth.token = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.534939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.535968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.536926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.537928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.538885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.539889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.540995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.541969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.542948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543667 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.543989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.544921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.545885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.546858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.547995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.548922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.549976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.550956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.551879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.552976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.553940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.554970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.555906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.556871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.557973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.558051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.558201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.558281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.558448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.558527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.559933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.560960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.561942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.562884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.563983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.564936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.565932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] role.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.566951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] role.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] role.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] role.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.567906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.568875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.569861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.570903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.571951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.572933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.573899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.574050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.574128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.574851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.574954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.575892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.576975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.577963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.578904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.579921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.connection = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.connection = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.580954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.581992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.582995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.583938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.584860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80346) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 14:19:38.584937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:19:38.585010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80347) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 14:19:38.615326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.615554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.615640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.615780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.615865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.616014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.624867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.625029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.653581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.653704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.653815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.653919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.797113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.797529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.846022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.846257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.846442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.846681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911559 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.911994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.912085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.912210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.912292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.937877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.937996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.938949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.939032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.939161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.939241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.939378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.950320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.950666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.962948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.963921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.964041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.964124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.964253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80347) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.964336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.964467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80346) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:19:38.982039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.982251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:19:38.982588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.982670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.982791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:19:38.982883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:19:38.983063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:19:38.983586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.983923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.984163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.984419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.994712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.994989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.995254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.995459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.995542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.995738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.996023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:38.996837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.004408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.005300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.007736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.008016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.008294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.008572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.008850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.008931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.009113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.009943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.010983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.011253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.011520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.037414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.038602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.039031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.039309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.040217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.040498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.045123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.045427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.045824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.046366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.046646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.047040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.053143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.053412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.054354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.054632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.057940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.058227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.058527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.059349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.059629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.059724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.059900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.059983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.060166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.060247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.060430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.060514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.060740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.061013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.061308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.061577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.079138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.079417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.080340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.080617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.084095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.085348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.087778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.088074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.088976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.089316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.093257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.094475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.096871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.098116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.100545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.100842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.101131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.101440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.101724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.101806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.102002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.102094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.102331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.102621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.102897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.103174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.122254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.122539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.122804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.123064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.123345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.123629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.123897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.124168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.131016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.132114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.132712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.132988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.133838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.134115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.136196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.136481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.136793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.137055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.137330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.137415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.137615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.138153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.138433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.138760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.150940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.151217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.151494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.151780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.152049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.152132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.152355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.152637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.152919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.153235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.153660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.153969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.154814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.155109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.172350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.173536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.175636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.176818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.178963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.180130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.180710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.181908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.182367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.182646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.182931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.183578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.184347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.184631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.190826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.191963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.192428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.193584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.195691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.195979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.196366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.196652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.196818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.197107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.197522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.197804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.205825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.206116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.206396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.206964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.207252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.207531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.214264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80347) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.214547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.215346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.215623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.215893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.215985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80346) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:19:39.216207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.216309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.216486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.216580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.216757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.216850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.217977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.218149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.218241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.218416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80347) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.218524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.218748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.219019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.219294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.219560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80346) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:19:39.256658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55a7a27079c0 pid: 80347 (default app) Jul 12 14:19:39.257949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55a7a27079c0 pid: 80346 (default app) Jul 12 14:19:39.260021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a3195-118b-4252-989b-604e40d68ae2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:39.260315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a3195-118b-4252-989b-604e40d68ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:39.260591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a3195-118b-4252-989b-604e40d68ae2 None None] PATH_INFO: `/v3` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:39.265134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Tue Jul 12 14:19:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.957896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.958442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.958442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.962736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:47.968389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.968831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.968952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.037273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: INFO dbcounter [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] Registered counter for database keystone Jul 12 14:19:48.057980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG oslo_db.sqlalchemy.engines [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80347) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 14:19:48.064095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writer thread running {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 14:19:48.238578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG passlib.handlers.bcrypt [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80347) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 14:19:48.238925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG passlib.handlers.bcrypt [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80347) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 14:19:48.293301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:48.295497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:48.413914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 14:19:48.588108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.658162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: INFO dbcounter [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] Registered counter for database keystone Jul 12 14:19:48.677466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG oslo_db.sqlalchemy.engines [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80346) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 14:19:48.683589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writer thread running {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 14:19:48.801748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59df5ca-d48b-4668-817d-2c11c23558a8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.806103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.806759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.807045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.807323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.815356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.867536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 14:19:48.867817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc014a4f-2549-4bda-92c1-3de137a31d0c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.877421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:19:48.883324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.898315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456afcd-981b-477d-9bd3-0773323653f4 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.902727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.904456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.904747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.905030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] PATH_INFO: `/v3/projects/27c596858f1e4aabb2a1fb3cbe092011` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.909125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] RBAC: Authorizing `identity:get_project(project_id=27c596858f1e4aabb2a1fb3cbe092011)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.959654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 14:19:48.960001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3817de-21cd-412c-8c2a-4480e81d5729 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.964498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/projects/27c596858f1e4aabb2a1fb3cbe092011 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:50.938436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9f45-3e37-4579-a393-99b8f3fe784e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.938943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9f45-3e37-4579-a393-99b8f3fe784e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.939041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9f45-3e37-4579-a393-99b8f3fe784e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.943144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:50.948340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.948759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.948905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.967407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:50.968909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:50.976737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 14:19:50.999375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec9f45-3e37-4579-a393-99b8f3fe784e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.011289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec9f45-3e37-4579-a393-99b8f3fe784e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.015788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.016419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.016707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.016990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.024599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.026423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17d952-be6b-4679-9851-5c3922a36340 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.035156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:19:51.040195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.049768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b36f18-62b0-4799-bc1e-1ac1d3afcf44 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.054997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.055283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.055561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.056947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.058819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4db856-a6c8-4c7f-9e80-556ffaf02c21 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.060989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.100585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482843c-d16a-4d42-981e-f354ed43efa6 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.101224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482843c-d16a-4d42-981e-f354ed43efa6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.101224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482843c-d16a-4d42-981e-f354ed43efa6 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.102516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:53.108144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.108523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.108699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.125809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:53.126964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:53.179843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b738-e6df-46ca-8dc4-b1617c3b67df None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.180135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b738-e6df-46ca-8dc4-b1617c3b67df None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.180425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471b738-e6df-46ca-8dc4-b1617c3b67df None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.181830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:53.187637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.187926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.188209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.208661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:53.228001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG passlib.handlers.bcrypt [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80346) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 14:19:53.228428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG passlib.handlers.bcrypt [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80346) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 14:19:53.231654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.254453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.274308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:53.275892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:53.332877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e3f7b-4676-4a7f-a591-ac9227a0a914 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.366693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.367406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.367688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.367964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.374443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:53.380656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:53.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.393795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:19:53.394645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:53.396561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.399580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.485772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.495971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbfcb7-e4e2-4932-a6ac-55e74079a990 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.498793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.503974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d418b-a9dc-4a2b-89db-7c05e4940485 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.534592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5142e51-3057-47e2-8486-615704db509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.535486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5142e51-3057-47e2-8486-615704db509d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.535771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5142e51-3057-47e2-8486-615704db509d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.536043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5142e51-3057-47e2-8486-615704db509d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.538128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.538787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.539069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.539346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.540097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.540399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.541864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.553419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:53.559143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.602337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5142e51-3057-47e2-8486-615704db509d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.603848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5142e51-3057-47e2-8486-615704db509d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.655157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5990c5-a7bc-4e8a-946f-e903e6827cbf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.688133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.689005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.689323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.689606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.690806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.692251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.693880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 9/16] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:53.698485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 8/17] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:19:53.699063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.704296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5142e51-3057-47e2-8486-615704db509d None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.706951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.713720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.797357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f2d60-be03-494e-b30d-c49c982750b1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.800073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5142e51-3057-47e2-8486-615704db509d None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.834149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.834952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.835240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.835514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.836684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.838231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.838630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.840191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.841824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.843413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:53.849228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.855589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.857620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:53.878743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.879645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.880459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:53.880459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:53.880459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:53.880459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:19:53.880459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:53.885643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.930055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.941762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.949706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9a3e-4d65-4bb5-9937-26a58016d781 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.983955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.984143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb072e-7680-47d8-a244-431bed98cf8e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.984789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.985140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.985449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.986288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.domain.id=default, user.id=cb516c8046b14d4c8dffdce40fe28f5d) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.986603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.988046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ca5bc-0a74-4905-96b2-2529d6af0645 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.994869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/role_assignments?user.id=cb516c8046b14d4c8dffdce40fe28f5d&scope.domain.id=default&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:54.019680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.020371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.020661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.020943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.021742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:54.022060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.023503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8965a-0975-40f5-8f11-5cba7025fcda None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.042818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:56.568435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb71ef-a47d-41cd-8004-50e3d0fe1741 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.569353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb71ef-a47d-41cd-8004-50e3d0fe1741 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.569818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb71ef-a47d-41cd-8004-50e3d0fe1741 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.571392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:56.576941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9fda-e711-432a-a498-c04b571da408 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.577818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9fda-e711-432a-a498-c04b571da408 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.578292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9fda-e711-432a-a498-c04b571da408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.602078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-95ef9fda-e711-432a-a498-c04b571da408 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:56.603561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-95ef9fda-e711-432a-a498-c04b571da408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:56.648770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb99e-f675-49b8-b798-80759a9673f3 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.649405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb99e-f675-49b8-b798-80759a9673f3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.649879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb99e-f675-49b8-b798-80759a9673f3 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.651719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 13/23] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:56.656863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.657385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.657871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.686157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:56.687723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:56.712411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 11/24] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:56.737949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94923285-7ff8-486b-b1c6-b31f420d4c3e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.738455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94923285-7ff8-486b-b1c6-b31f420d4c3e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.738882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94923285-7ff8-486b-b1c6-b31f420d4c3e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.740621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 12/25] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:56.748625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18690c2e-f045-485f-b788-6bd05fe0b32a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.749257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18690c2e-f045-485f-b788-6bd05fe0b32a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.749753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18690c2e-f045-485f-b788-6bd05fe0b32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.786755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-18690c2e-f045-485f-b788-6bd05fe0b32a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:56.789118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-18690c2e-f045-485f-b788-6bd05fe0b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:56.818939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 14/26] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:56.820231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.867433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.896552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 13/27] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:56.906482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85762e-8d02-4069-b11c-f921862a543c None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.906983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85762e-8d02-4069-b11c-f921862a543c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.907540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85762e-8d02-4069-b11c-f921862a543c None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.909510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 14/28] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:56.910254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85762e-8d02-4069-b11c-f921862a543c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.926133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.943703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d3852-82e2-42e4-8adc-b871583edf1f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.980007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.981482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.981944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.982408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.993124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.994817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-09cac9a0-646e-44ac-8d3c-783e43d08950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:57.014680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:57.015491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:57.016383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:57.016383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:57.016383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:57.016383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:19:57.018326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 15/29] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:57.019816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.020278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.020739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.038473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:57.039875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:57.045028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85762e-8d02-4069-b11c-f921862a543c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.081304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.082351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.082783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.083267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.095676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:57.104889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:57.106153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:57.107008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:19:57.114336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 16/31] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:57.114336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.114557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 15/31] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:57.114557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.172532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.175716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.221677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2636983-0bff-4349-8aa9-8180c1a9dabb None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.242154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585afda-de4f-4c31-8118-9102b2ef5d8b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.258496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.259233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.259526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.259804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.260605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.261978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.282857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.283618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.283899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.284242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.284980 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.285326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.286724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.298615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 16/32] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.299216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.415837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.422617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbf5e8-de9b-413d-84df-fbe7cdaba6eb None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.459931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.460937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.461466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.461987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.463872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.464432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.466201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.479731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 17/33] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.480945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.484889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 17/34] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Jul 12 14:19:57 2022] POST /identity/v3/roles => generated 225 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:57.485988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.607482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6d65d-6ddd-4604-9423-1dd7f6970fbf None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.651100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.652300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.652737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.653219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.659440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.659440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.725277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbce1f1-9933-4409-a95d-b6e4f20f05cf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.744277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.764829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.765532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.765819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.766096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.767097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.768405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d826f8d-5563-4d36-aaad-6681b03ed289 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.875101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 18/35] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Jul 12 14:19:57 2022] POST /identity/v3/roles => generated 219 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:57.876067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.884952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 18/36] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:19:57 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:57.899225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2e13-04d8-45b6-b0e8-2d91c75d0b55 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.899540 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2e13-04d8-45b6-b0e8-2d91c75d0b55 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.899808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2e13-04d8-45b6-b0e8-2d91c75d0b55 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.900281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.913552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 19/37] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:57 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cbed-a49d-4322-a4f5-5acb24dde702 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cbed-a49d-4322-a4f5-5acb24dde702 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2cbed-a49d-4322-a4f5-5acb24dde702 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 20/38] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd07f-8f66-43f1-9541-6f24176b332d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.920982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd07f-8f66-43f1-9541-6f24176b332d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.921552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd07f-8f66-43f1-9541-6f24176b332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.947538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-199fd07f-8f66-43f1-9541-6f24176b332d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:57.954358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-199fd07f-8f66-43f1-9541-6f24176b332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:58.014439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00c0c1-cad2-4d2e-b87c-7d1d71655d39 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.056172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 21/39] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:58.056355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.057297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.057588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.057983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.058638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.059683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.060164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.060363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.063261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.067536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 19/40] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:58.070591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.096499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:58.097840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:58.174954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 22/41] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:58.175745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.179731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.194755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcff95-d4f0-4e46-a0f1-74e8e03c4719 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.211499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.229194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.230034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.230323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.230599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.231899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.233380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.235174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 20/42] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:58.235762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.284817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c703-28f8-4aa9-9d81-3cb7ea6f4d3f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.337384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dca50-69dd-4c01-b300-3269d65d87af None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.373325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.374009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.374297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.374579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.375356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.376792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.424442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.425292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.425534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.425821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.427100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.428613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.430316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 23/43] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:58.431113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.449002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 21/44] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Jul 12 14:19:58 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:58.450537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.482175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.539125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.546212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be76d8-e39f-44e0-866b-231d0e2d815b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.572407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c411d-05e9-4e05-a278-eedb60a16cf4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.584109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.585054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.585379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.585683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] PATH_INFO: `/v3/domains/default/users/cb516c8046b14d4c8dffdce40fe28f5d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.587873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cb516c8046b14d4c8dffdce40fe28f5d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.589452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0185ff9-52cf-4a29-ab95-d3ceda9c8e95 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.611305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.611983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.612276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.612687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.613447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.614911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc865d-c099-4099-87fb-afcdee57708f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.654989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 24/45] 10.222.0.30 () {58 vars in 1361 bytes} [Tue Jul 12 14:19:58 2022] PUT /identity/v3/domains/default/users/cb516c8046b14d4c8dffdce40fe28f5d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:58.733752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 22/46] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:19:58 2022] POST /identity/v3/projects => generated 312 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:00.680758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9887-0976-4c94-814a-d0078e1d2e63 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.681319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9887-0976-4c94-814a-d0078e1d2e63 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.681404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf9887-0976-4c94-814a-d0078e1d2e63 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.682899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 25/47] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:00.688443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.688819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.689110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.705897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:00.707331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:00.756821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13be2b-c116-4fa7-822f-1176a7151709 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.757122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13be2b-c116-4fa7-822f-1176a7151709 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.757447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13be2b-c116-4fa7-822f-1176a7151709 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.758818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 26/48] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:00.764153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.764452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.764742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.767790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.775367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 23/49] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:00.781189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:00.782385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:00.798095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.849227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 27/50] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:00.873060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.896367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa325e-6dcb-48eb-87cb-d8b7b3f5d18f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.931138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.931932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.932221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.932503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.942969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:00.951186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:00.952037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:00.952808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 24/51] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:00.957349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.958353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.972573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b516a-42f9-409a-a72d-f0b705c7e779 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.009832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.010656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.010951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.011232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.013403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.014915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.016635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 28/52] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.022594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.117452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.123438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda00f3-8b14-4d50-8d47-a5488e325e08 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.157344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.158015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.158307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.158586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.159316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.160777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.161408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c199-8456-4fd0-bc26-6260fb376921 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.194255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.194919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.195209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.195494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.196246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.196559 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.197977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.207073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 25/53] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.212606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.255305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 29/54] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Jul 12 14:20:01 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:01.318058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5cd83-5b47-45b2-92e3-579ae51a675a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.336626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.354057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.354946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.355238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.355517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.357415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.358888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.360749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 26/55] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:01.366254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.376505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.469704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334117b1-8d94-4dff-9bd9-4ff4f633b1a4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.504318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.505147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.505455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.505734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.506898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.508394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edf5d3-8173-4351-9a02-0952df68e951 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.510029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 30/56] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.515930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.614382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e81a-904b-4f9b-9316-d5193e197881 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.648288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.648942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.649258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.649538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.650377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.domain.id=default, user.id=cb516c8046b14d4c8dffdce40fe28f5d) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.650691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.652127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c6f03-d8a2-4e6c-a94b-86df0500df9c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.661437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 27/57] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/role_assignments?user.id=cb516c8046b14d4c8dffdce40fe28f5d&scope.domain.id=default&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:03.274459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6d49-97aa-4da2-88ef-38e6451c23b3 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.274977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6d49-97aa-4da2-88ef-38e6451c23b3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.275088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6d49-97aa-4da2-88ef-38e6451c23b3 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.276378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 31/58] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:03.282011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.282383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.282580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.299560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:03.300739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:03.344671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:03.364212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 28/59] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:03.387734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a6d49-97aa-4da2-88ef-38e6451c23b3 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.461840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:03.489904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a6d49-97aa-4da2-88ef-38e6451c23b3 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.523312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.524093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.524373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.524693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.525819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:03.527274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71cffe-61b8-4ad7-afa6-424fba214d52 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:03.529489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 32/60] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:03.534946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.628897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357c39-762c-44ab-bf8b-18e860fcbe13 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.662152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472debf-c264-4039-a97b-36834b994463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.662811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472debf-c264-4039-a97b-36834b994463 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.663099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472debf-c264-4039-a97b-36834b994463 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.663380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472debf-c264-4039-a97b-36834b994463 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.664095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472debf-c264-4039-a97b-36834b994463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:03.665570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472debf-c264-4039-a97b-36834b994463 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:03.770923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 29/61] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:03 2022] POST /identity/v3/users => generated 275 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:06.884619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e5c8-c091-4cbe-808c-c1fdcf0a559d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.885638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e5c8-c091-4cbe-808c-c1fdcf0a559d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.886142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e5c8-c091-4cbe-808c-c1fdcf0a559d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.888688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 33/62] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:06.894055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c7e7-aba1-4ab4-a939-352d94bc803d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.894452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c7e7-aba1-4ab4-a939-352d94bc803d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.894612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c7e7-aba1-4ab4-a939-352d94bc803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.916542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d6f5c7e7-aba1-4ab4-a939-352d94bc803d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:06.918058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d6f5c7e7-aba1-4ab4-a939-352d94bc803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:06.954466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.011837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 30/63] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:07.039531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2aac-6492-4ed4-b29f-618112fc9c7f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.040163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2aac-6492-4ed4-b29f-618112fc9c7f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.040648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f2aac-6492-4ed4-b29f-618112fc9c7f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.042250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 34/64] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:07.047651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cb65-3c22-4918-ba3d-bd81c515079c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.047954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cb65-3c22-4918-ba3d-bd81c515079c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.048248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cb65-3c22-4918-ba3d-bd81c515079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.049670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f2aac-6492-4ed4-b29f-618112fc9c7f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.076556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0d10cb65-3c22-4918-ba3d-bd81c515079c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:07.079822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0d10cb65-3c22-4918-ba3d-bd81c515079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:07.171473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 31/65] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:07.172891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf55f-b56f-4c57-bd94-a7a07eeafc85 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.173468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf55f-b56f-4c57-bd94-a7a07eeafc85 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.173896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf55f-b56f-4c57-bd94-a7a07eeafc85 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.175637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 32/66] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:07.181479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.182005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.182470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.194986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f2aac-6492-4ed4-b29f-618112fc9c7f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.203908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:07.205218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:07.243603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.244649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.245077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.245521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.259245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:07.270109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.271166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:07.272122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:07.274968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 35/67] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:07.275958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.278841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.296397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 33/68] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:07.297716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.340441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.402830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c683155-f57c-43a2-b87c-3996915ca948 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.440146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.440952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.441266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.441543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.443854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ca73-d229-4ae6-a570-7fae08e265ac None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.453520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:07.461123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.462028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:07.462792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 36/69] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:07.463459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.485380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.486055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.486346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.486628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.487385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.487695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.489095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350df83-eeec-4f2c-af6d-29a0fe4e9cc6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.498308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 34/70] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.503938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303214c4-8948-4a50-a1b4-7b61946c8fa9 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.504234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303214c4-8948-4a50-a1b4-7b61946c8fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.504518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303214c4-8948-4a50-a1b4-7b61946c8fa9 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.509584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 35/71] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:07.509584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303214c4-8948-4a50-a1b4-7b61946c8fa9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.532095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.574281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99340685-dba0-4439-8232-38c4a2fe261a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.601536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.607334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.608171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.608464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.608762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.614386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303214c4-8948-4a50-a1b4-7b61946c8fa9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.619203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:07.627828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.628703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:07.629500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 37/72] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:07.630191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.653883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.654551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.654828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.655113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.655866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.656178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.657601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e4da7-b1df-477c-aef1-46a19e169867 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.665444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 36/73] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.666598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf598a-af9d-491b-85f6-0c0764edc972 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.666887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf598a-af9d-491b-85f6-0c0764edc972 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.667170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf598a-af9d-491b-85f6-0c0764edc972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.685234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-33bf598a-af9d-491b-85f6-0c0764edc972 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:07.686426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-33bf598a-af9d-491b-85f6-0c0764edc972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:07.726510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ce1b4-8f2d-44dd-b8d6-ea304baf2715 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.748322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.750250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 37/74] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:07.751674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939725d-ed88-407f-bb25-52dffae3d653 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.751969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939725d-ed88-407f-bb25-52dffae3d653 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.752249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939725d-ed88-407f-bb25-52dffae3d653 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.757725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 38/75] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:07.757725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939725d-ed88-407f-bb25-52dffae3d653 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.764405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.767407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.767407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.767407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.781326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.782791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.788770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 38/76] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.788770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.837424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.858576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939725d-ed88-407f-bb25-52dffae3d653 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.883723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b552-1705-49e0-96db-9f9cf676ace8 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.891265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.891936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.892225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.892505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.893292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.893605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.894992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793a381-550a-4f94-b21f-c6c6baa97dd3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.903229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 39/77] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.904384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.904675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.904958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.917599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.919088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.919367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.919642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.920837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.921042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:07.922241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:07.922344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.923907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 39/78] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.924657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.971719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.991909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 40/79] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:07.992756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.023641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162836f-ead4-4189-b2b1-c4fec6d30d11 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.056885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.057713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.057996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.058271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.059904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.061370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.062492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.067293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 40/80] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:08.067293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.091488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925085a-03a7-4171-a8a4-f85d77991c31 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.126094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.126744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.127029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.127306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.163175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32940f-3879-46ba-8b98-959916f8fe1a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.178712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.193140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.194687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.196126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.196989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.197301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.197579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.198783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.200728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.202513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 41/81] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:08.204437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.262469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.292643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 41/82] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:08.293535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.301874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966c4d2-acb9-4493-8a5e-3bc2e283ab71 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.337319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.337969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.338248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.338523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.388626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.390351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab97c0-fdef-4413-8eb7-c3d4492043e0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.404194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.405862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.424904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.425735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.426239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.426344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.437199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:08.444829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:08.445909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:08.446714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 42/83] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:08.447363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.477912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.506663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 42/84] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:08.507501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.549794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d25072-4bbf-46d4-8251-dae4931f75b1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.584067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.584734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.585019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.585337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.586055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.587657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.589083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.604146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5ab9-db61-415a-8eec-f2dfd9d2bb77 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.643602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.644723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.645272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.646103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.662499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 43/85] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:08 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:08.663546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.730343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.731863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.763752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.777075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a195-a8e7-489e-9c9e-95b3099f17b8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.815578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.816407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.816687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.817504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.819385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.820993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.825370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 44/86] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:08.826208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.829535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.849100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 43/87] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:08.849991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.941942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2dd69-ba6c-4c64-99da-a5fdfa024dbc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.971966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6498-eda2-4b1c-8f1d-244975ecbf38 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.983407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.984086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.984377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.984659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.985466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:08.985785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.987184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.996092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 45/88] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:08.997639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.015962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.016746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.017031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.017544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.019819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.020101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.025208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 44/89] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.025208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.029885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.101682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.129198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b466d5-8307-4f71-9d74-e3252aceb656 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.144754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f25021-6e28-46f9-a493-fd16bbc70c33 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.176211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.177733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.178170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.178834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.179861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.180325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.181961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.182770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.183767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.184155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.184571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.186152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.187545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.189089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 46/90] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=0571188a7619422aac6f72fee883d8d6 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.194401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 45/91] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.194401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.195592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.295467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.309300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62378a0b-3504-4e5f-a72a-3f4ea3262f22 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.316996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264a67e-6b3f-4130-811d-313b4e824f02 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.331133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.363430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.364394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.364860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.365343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.367268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=cb516c8046b14d4c8dffdce40fe28f5d) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.368560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.370671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.378779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.380019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.380483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.380938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.383537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 46/92] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/role_assignments?user.id=cb516c8046b14d4c8dffdce40fe28f5d&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.384607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.386436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.388050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.391586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 47/93] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.396655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.608752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b362e4-bd68-4ee6-b2d4-0debed113f91 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.651084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.651799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.652096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.652377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.679995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d6f9-0783-4460-8345-ac7fcf8f9687 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.689518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.723584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.724250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.724536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.724856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.725803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.726120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.730278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.744077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 47/94] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.761868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.763741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.877322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 48/95] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:09.882610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.949513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.013121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc62fa-a785-4a95-83ad-67212c10fa70 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.062787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.063660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.063944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.064219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] PATH_INFO: `/v3/projects/160e05ed19894d30829fc6a0e85de880` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.065474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] RBAC: Authorizing `identity:get_project(project_id=160e05ed19894d30829fc6a0e85de880)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.066934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d32eb-4d29-46f5-b9ff-71fb44b8e815 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.068632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 48/96] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/projects/160e05ed19894d30829fc6a0e85de880 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:10.074453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.138074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.200485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a856468-254b-4517-8e90-fca61c28ecd4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.243712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.244376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.244662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.244943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.245816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=160e05ed19894d30829fc6a0e85de880, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:10.246131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.248107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b08f15-1631-4f09-a0dd-e3a5043b5a3e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.257251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 49/97] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=160e05ed19894d30829fc6a0e85de880&role.id=0571188a7619422aac6f72fee883d8d6 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.180676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84a745-0ea1-4c88-bc3b-d273ba4961fc None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.181220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84a745-0ea1-4c88-bc3b-d273ba4961fc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.181300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84a745-0ea1-4c88-bc3b-d273ba4961fc None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.182902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 49/98] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:11.187949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.188416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.188502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.211930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:11.213142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:11.302798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 50/99] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:11.338673 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84a745-0ea1-4c88-bc3b-d273ba4961fc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.449090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84a745-0ea1-4c88-bc3b-d273ba4961fc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.466311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.483920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.484727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.485010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.485315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.496782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:11.504346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.505259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:11.506418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 50/100] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.511610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.567048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.650811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e8883-77d4-4d62-bf54-0901cb4df9fb None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.695959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.696547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.696831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.697111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.697891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:11.698201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.700137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4ab6a-6d07-477f-a652-5f5f22ba8102 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.712346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 51/101] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.717848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.796161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abad1b-ea28-499c-a782-76bfaefcbd6f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.796487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abad1b-ea28-499c-a782-76bfaefcbd6f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.796725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abad1b-ea28-499c-a782-76bfaefcbd6f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.798184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 52/102] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:11.803291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.803582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.803862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.830881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:11.832072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:11.867871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dddb2-327a-4bed-b554-e289ffc43652 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.939235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 53/103] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 14:20:11.964595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.969864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.970764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.971056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.971341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] PATH_INFO: `/v3/users/98f7e7d4e99b4daa83470756a399baa1` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.003786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] RBAC: Authorizing `identity:get_user(user_id=98f7e7d4e99b4daa83470756a399baa1)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.005330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.007390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 51/104] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/users/98f7e7d4e99b4daa83470756a399baa1 => generated 289 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.009584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.018167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.078075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933f1a-5e2f-4675-82e5-f7981674ac87 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.115774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.148854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.149696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.149982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.150898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.152104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.154291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.156567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 54/105] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.162938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.253904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8817-40a7-48ef-bf82-66a2a038400f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.298959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a2200-83dd-4a08-a01f-836dc932a78a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.309524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.334409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.334747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.335079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.335370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.335652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.336247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.336410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.336869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.337377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.337852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936c1cf-23b6-48a9-afaf-8654975d7d01 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.420598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.422160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.461802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.462840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 55/106] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:12 2022] POST /identity/v3/projects => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:12.464179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637e1b4-e3f7-476b-8285-fce17527c75d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.464505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637e1b4-e3f7-476b-8285-fce17527c75d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.465343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637e1b4-e3f7-476b-8285-fce17527c75d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.466815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 56/107] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:12.478749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.479048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.479326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.514227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:12.515361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:12.562494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 52/108] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:12.568085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.684508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9e63-a092-4a14-b3f3-e9ede1b20693 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.716199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 57/109] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:12.724090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.724953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.729194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.729194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.730164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.732310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.734192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 53/110] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:12.740066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.744253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.747910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.895966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06234e-6281-4615-b123-9a6d286f1053 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.909638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.932251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844c6e-4cf2-4b90-92fd-e602f92d8aa2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.951807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.952846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.953344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.953802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.955063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=98f7e7d4e99b4daa83470756a399baa1) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.955635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.957317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c0b9-6a22-49ef-8b96-ba6350df75e1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.969835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 58/111] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/role_assignments?user.id=98f7e7d4e99b4daa83470756a399baa1&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=8185353471b64ed18c802551c3ad37a0 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.971529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c96bd-aad3-4385-a8c4-40a47fef5127 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.972108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c96bd-aad3-4385-a8c4-40a47fef5127 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.980111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c96bd-aad3-4385-a8c4-40a47fef5127 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.980111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 59/112] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:12.982931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb0e4-f65a-46d4-88a4-f215ed9f8e47 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.983370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb0e4-f65a-46d4-88a4-f215ed9f8e47 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.983890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eb0e4-f65a-46d4-88a4-f215ed9f8e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.006289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.007115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.007397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.007676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.035124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-835eb0e4-f65a-46d4-88a4-f215ed9f8e47 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:13.036332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-835eb0e4-f65a-46d4-88a4-f215ed9f8e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:13.038083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:13.058161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:13.059667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:13.060456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 54/113] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:13.069226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.172263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 60/114] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:13.173626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242b6d3-ef25-4b2a-b807-5b5db277ea6a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.173927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242b6d3-ef25-4b2a-b807-5b5db277ea6a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.174213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242b6d3-ef25-4b2a-b807-5b5db277ea6a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.175728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 61/115] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:13.176694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab053-4352-4322-9ad2-8382b6221193 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.176982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab053-4352-4322-9ad2-8382b6221193 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.177309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbab053-4352-4322-9ad2-8382b6221193 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.178776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 62/116] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:13.186711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3cb66-9854-45f4-923e-cde6797b4df3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.187030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3cb66-9854-45f4-923e-cde6797b4df3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.187284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3cb66-9854-45f4-923e-cde6797b4df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.206262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4bd3cb66-9854-45f4-923e-cde6797b4df3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:13.207495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4bd3cb66-9854-45f4-923e-cde6797b4df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:13.228551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.272374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91310b56-86aa-4cd6-bf73-844d099bff48 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.297020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 63/117] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:13.298736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.299200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.300079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.320564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:13.322209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:13.325310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.329980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.330667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.330953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.331239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.331993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.332302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.333776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.345144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 55/118] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.346043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.410310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 64/119] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:13.411480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.521066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e070-0a43-487c-9ee5-426b497fdee5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.552354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.569436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.570488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.570895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23e70-f273-4ebf-ab1d-bd36451e9043 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.571069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.571509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.586069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:13.596644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:13.597734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:13.598660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:13.606391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 56/120] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:13.606986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.619359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.620176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.620462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.620741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.631381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.635014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:13.646045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:13.647847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 65/121] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:13.648629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.740360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d4c4-701e-4b09-a822-5d39fdcc3819 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.773357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c8d2-5afc-4a55-9db6-af6d97e14c3b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.784559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.786053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.786340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.786620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.788985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.791452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.794120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 57/122] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.794914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.818957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.820111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.820600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.821124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.833285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:13.842391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:13.843264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:13.844039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:13.846540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 66/123] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:13.852177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.853229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.903518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c245-ca5f-4cad-9cab-10a094d6292d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.916811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.941281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.942002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.942285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.942619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.943418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.943725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.945214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.958794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 58/124] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.958794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.970995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c5a4-f295-4fc4-b64f-6e67f5fa16a6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.009611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.010712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.011401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.012408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.013642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:14.014153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.015949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.025485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 67/125] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.026527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.063521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62537bc3-1d2e-4614-beb5-1f45ed93b1cf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.078230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.099214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.101282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.101282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.101282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.102195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.103971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.106031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 59/126] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.107016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.137751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b2a40-2a23-47e3-ae88-b4d176a7d77a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.162702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.179049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.180024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.180320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.180600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.181388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:14.181951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.184254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.194847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 68/127] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.196277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.222683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0530fa-11a8-4116-9048-88f32bb61afa None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.259603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.260584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.260942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.261249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.263351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.264796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.266823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 60/128] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.267543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.305858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38991655-59e1-4ead-ab2c-d3330814ae30 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.321120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.343540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.344874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.345393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.345870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.347320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.348970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.350902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 69/129] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.351866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.375874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703517-82b1-4cc2-bfb2-c3ab4f6275ea None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.407248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.414727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.415882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.416308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.416725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/cb516c8046b14d4c8dffdce40fe28f5d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.419097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user_id=cb516c8046b14d4c8dffdce40fe28f5d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.420686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.459617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcd96-778b-4e50-ae65-f53790557a35 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.494313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.495175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.495459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.495777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.497579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.499019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.500670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 70/130] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.505274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.516664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 61/131] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:14 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/cb516c8046b14d4c8dffdce40fe28f5d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:14.518259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.668677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079bf0b-6296-4877-bf43-76763b59b889 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.697531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.717655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.718536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.718820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.719108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.720327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.721900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.724545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 71/132] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.725046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb1249-0f62-45b8-8587-dfe3f61834ce None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.725424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.739021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.766209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.767574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.768057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.768530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.769983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.771628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.774381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 62/133] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.776201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.861252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7c9f4-4b03-40b6-88dd-d3c3431ab448 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.896923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192df5f-5348-4508-91bc-e287c6fc2438 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.905458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.906914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.907206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.907491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] PATH_INFO: `/v3/projects/160e05ed19894d30829fc6a0e85de880` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.908734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] RBAC: Authorizing `identity:get_project(project_id=160e05ed19894d30829fc6a0e85de880)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.911136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.913499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 72/134] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/projects/160e05ed19894d30829fc6a0e85de880 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.914596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.943679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.944637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.944927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.945373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/fa38071580e54275a3c6d76058090878/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.948012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user_id=fa38071580e54275a3c6d76058090878, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.949971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.989541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.032659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac7a34-a200-49ca-a880-a8b15a79c24e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.070939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.072207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.072732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.073331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/fa38071580e54275a3c6d76058090878/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.076215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user_id=fa38071580e54275a3c6d76058090878, role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.076703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 63/135] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:14 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/fa38071580e54275a3c6d76058090878/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:15.077241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.078674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33a44-f94f-47a2-b7c1-0644154978b4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.143031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.169940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 73/136] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:14 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/fa38071580e54275a3c6d76058090878/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:15.171943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ded2-b1f5-40c8-a053-c72cc4a421ed None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.172492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ded2-b1f5-40c8-a053-c72cc4a421ed None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.172973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863ded2-b1f5-40c8-a053-c72cc4a421ed None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.176933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 74/137] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:15.183844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b42-2549-4838-89a4-67516448ad3f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.184529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b42-2549-4838-89a4-67516448ad3f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.185036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b42-2549-4838-89a4-67516448ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.230177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-550f0b42-2549-4838-89a4-67516448ad3f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:15.232581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-550f0b42-2549-4838-89a4-67516448ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:15.269654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bca50-51cf-48dc-ad6d-a2d1ceb14a20 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.316205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.317555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.318043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.318519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] PATH_INFO: `/v3/projects/160e05ed19894d30829fc6a0e85de880/users/fa38071580e54275a3c6d76058090878/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.322488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=160e05ed19894d30829fc6a0e85de880, user_id=fa38071580e54275a3c6d76058090878, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.325197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 75/138] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:15.326292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.342753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ed26-a0c8-4681-bd6a-c98a8dfa1c4a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.343346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ed26-a0c8-4681-bd6a-c98a8dfa1c4a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.343857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69ed26-a0c8-4681-bd6a-c98a8dfa1c4a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.347896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 76/139] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:15.352918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.353229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.353509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.373658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:15.375259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:15.411692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 64/140] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:15 2022] PUT /identity/v3/projects/160e05ed19894d30829fc6a0e85de880/users/fa38071580e54275a3c6d76058090878/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:15.412622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.430043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.475793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 77/141] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:15.514056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.593552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.593552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e391cb-f4eb-42d4-9d93-a756f1978cd2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.632822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.633672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.634022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.634308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.635515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.637499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.639806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 65/142] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:15.647168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.710691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e4985-bfda-4df7-adbc-6c7e0ff04438 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.755232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.756041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.756332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.756615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.772103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f03578-ba51-46ff-82d3-d5727f31916e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.782053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:15.790350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:15.791260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:15.792051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 78/143] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:15.812052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.812739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.813029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.814348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.815113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.815550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.816859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.870535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.940810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 66/144] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:15 2022] POST /identity/v3/users => generated 283 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:15.957997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532509e-fadc-4e54-844b-cb150b2f495a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.003267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.004028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.004316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.004598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.005535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:16.005849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.007354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.020939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 79/145] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:16.026652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.057956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.144688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb168c2-7756-469b-a68f-dbeac368e9a2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.181762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.184189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.184189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.184189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] PATH_INFO: `/v3/users/98f7e7d4e99b4daa83470756a399baa1` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.185109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] RBAC: Authorizing `identity:get_user(user_id=98f7e7d4e99b4daa83470756a399baa1)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.186629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.188344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 67/146] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/users/98f7e7d4e99b4daa83470756a399baa1 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:16.194000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.315415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.319936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7597-91b0-4dfd-bc76-c01c17843717 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.354339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.355257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.356117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.356466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.357681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.359227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d4de4-51fa-4a73-8911-a40f5799bb60 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.360938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 80/147] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:16.366701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.468879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbfec8-3232-4554-967c-4e0f2ec79465 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.507987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.508888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.509191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.509535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/98f7e7d4e99b4daa83470756a399baa1/roles/8185353471b64ed18c802551c3ad37a0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.511559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user_id=98f7e7d4e99b4daa83470756a399baa1, role_id=8185353471b64ed18c802551c3ad37a0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.513605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a149f40-024c-4441-b834-b94b37746ff3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.606789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 68/148] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:16 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/users/98f7e7d4e99b4daa83470756a399baa1/roles/8185353471b64ed18c802551c3ad37a0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:17.428103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa2892-669d-409e-be13-8519f6722b68 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.428103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa2892-669d-409e-be13-8519f6722b68 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.428103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa2892-669d-409e-be13-8519f6722b68 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.429357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 81/149] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:17.436710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.437473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.437473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.456391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:17.459468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:17.537905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 69/150] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:17.573188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa2892-669d-409e-be13-8519f6722b68 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.688370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa2892-669d-409e-be13-8519f6722b68 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.733309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.734122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.734403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.734686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.753279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:17.761113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:17.762563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:17.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 82/151] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:17.770775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.878055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.898811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2594c-d07b-4223-ae9a-99a424604776 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.935836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.936527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.936813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.937091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.937896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.938203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.939661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.953176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 70/152] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.959255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.015776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.171568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52a3b5-b1a1-4eee-af50-37b54ba7c781 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.216425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.217224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.217512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.217789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] PATH_INFO: `/v3/users/cb516c8046b14d4c8dffdce40fe28f5d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.219006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] RBAC: Authorizing `identity:get_user(user_id=cb516c8046b14d4c8dffdce40fe28f5d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.220540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb8e0-fd70-413a-b981-080f1d1de3af None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.222560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 83/153] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/users/cb516c8046b14d4c8dffdce40fe28f5d => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:18.228059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.377511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a78462-18b2-4523-9581-fde76a004b1d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.425063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.425746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.428540 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.428540 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.448720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.525131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.526566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.528995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266644-fc1c-4709-9bc2-6e5c047f76aa None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.529486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266644-fc1c-4709-9bc2-6e5c047f76aa None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.529569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266644-fc1c-4709-9bc2-6e5c047f76aa None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.531221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 84/154] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:18.544944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.545260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.545549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.582071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:18.583439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:18.704923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 85/155] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:18.746466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.755720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 71/156] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:18.764282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.794463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.860523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.955068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d4176-e70a-403c-8984-df058c1d1c32 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.989854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cdd99-2ce0-41cf-94e3-b22dc38c47c0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.033414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.034254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.034788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.035073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.066045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-5eb707ba-acc1-43cb-bdee-f057030a1545 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:19.074635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:19.075582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:19.076760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 86/157] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:19.077458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.077554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7823986-22f0-49f7-8d6f-f1ed39b7fedd None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.077554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7823986-22f0-49f7-8d6f-f1ed39b7fedd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.077731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.077808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7823986-22f0-49f7-8d6f-f1ed39b7fedd None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.078164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.078164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.079288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 87/158] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:19.079477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.080370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b7296-14b2-466d-9dba-8901efaaa441 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.080652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b7296-14b2-466d-9dba-8901efaaa441 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.080843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.081001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b7296-14b2-466d-9dba-8901efaaa441 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.082814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 72/159] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:19.086416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 88/160] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:19.090782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.091001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.091271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.091653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.120336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:19.121763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:19.223496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 89/161] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:19.224408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.244715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.339394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426ee3-47f4-4cb4-996c-a17aac61acc8 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.365518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.411405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.412087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.412374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.412653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.413441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:19.413752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.415134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdeddf-b509-46b6-bdeb-e199518c373b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.429281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 73/162] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:19.430760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.431058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.431345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.460079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba2e97-b6f0-43e6-b98c-bf45b80bee1b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.463621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:19.465536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:19.585328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 74/163] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:19.586365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.597955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.598585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.598839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.599115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.599969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=cb516c8046b14d4c8dffdce40fe28f5d) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:19.600281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.601777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.619595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 90/164] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/role_assignments?user.id=cb516c8046b14d4c8dffdce40fe28f5d&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:19.620345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.731451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1122c-1b04-43b1-8c64-cb17b6fd0606 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.733064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.764818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.790989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.791872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.795303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.795303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.800234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93db4f-34c1-4863-9314-5cfc4c7a8985 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.810345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:19.818326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:19.819235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:19.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 75/165] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:19.820885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.890022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.890832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.891115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.891393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.898103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.899592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.901734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 91/166] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:19.903050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.002098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152eef2-ee71-4e8c-9041-59d64051391d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.046725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.047532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.047818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.048155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.058754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:20.065947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8de2cf-305c-4d94-a5bd-82f8c725ec34 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1c84bdca-3441-4b0e-a57d-2f69ae786cde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:20.066582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:20.067456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:20.068252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 76/167] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:20.068990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b8a64-db2b-41cb-999e-336b6eab615a None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.069800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b8a64-db2b-41cb-999e-336b6eab615a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.069800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b8a64-db2b-41cb-999e-336b6eab615a None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.070991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 77/168] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:20.071613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b8a64-db2b-41cb-999e-336b6eab615a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.099139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.103479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.104151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.104434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.104707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.105516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:20.105823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.107213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.117423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 92/169] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.118316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.144374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.189740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b8a64-db2b-41cb-999e-336b6eab615a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.227543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.227780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7d1bf-6471-4c1d-ad94-884e002cf338 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.228214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.228501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.228779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.267505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.268331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.268712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.269103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.269983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:20.270389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.271869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089ff13-4302-4140-bb17-a94a60f7f705 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.282160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 93/170] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.283682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.284072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.284449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.301775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:20.303117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:20.311275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.313250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.344658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.372082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.377034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 94/171] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:20.378223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.468272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 78/172] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:20.469074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.535171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ae06-7bc8-4d09-89b8-da053e68ae3b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.570298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.571173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.571456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.571734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.573125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.575335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.576459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2757-b545-4430-8c93-00e9d4c29779 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.577353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 95/173] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.578444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.601478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.613640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.614658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.615002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.615370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] PATH_INFO: `/v3/users/fa38071580e54275a3c6d76058090878` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.616864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] RBAC: Authorizing `identity:get_user(user_id=fa38071580e54275a3c6d76058090878)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.618542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.620527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 79/174] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/users/fa38071580e54275a3c6d76058090878 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.621876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.660597 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.705276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641db2a-55d4-4ad9-a153-ef1ec3005a38 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.732932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112d8c9-66ff-404a-a9fb-b0a249ad6c37 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.750279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.751652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.752491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.752941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.765703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:20.771433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.772515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.772968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.773441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-4a92fe15-4a39-4bcc-a39f-a2703e8d015e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:20.774224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:20.775107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:20.775887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:20.776698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 96/175] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:20.777681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.778141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d125-8430-4a18-b544-c000cee19e6c None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.778497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d125-8430-4a18-b544-c000cee19e6c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.778931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d125-8430-4a18-b544-c000cee19e6c None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.779923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.782004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 80/177] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.783038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.784565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 97/177] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:20.785631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286d125-8430-4a18-b544-c000cee19e6c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.854414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.901854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28932145-b40b-419e-af2c-49d7431a7a37 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.907649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286d125-8430-4a18-b544-c000cee19e6c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.942184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.942507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.943536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.943830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.944110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.950295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.953350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.953350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.953350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.009977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.011882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.019350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.020808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594984cc-194a-447a-a785-ee471fae0860 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.066276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.109089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 81/178] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:21.109954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.126541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 98/179] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:21.126541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.126541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.126541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.140955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:21.142182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:21.164623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.207661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a237c-68e8-4ce3-a3ad-9d394aaa4ad7 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.208507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 99/180] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:21.209364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.248049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.248714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.248994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.249299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.250047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.250348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.251711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2d90f-42b2-4a66-8171-753fa86b6be3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.259599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 82/181] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.260837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea530-79b8-498c-9834-49de38bfccf5 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.261066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea530-79b8-498c-9834-49de38bfccf5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.261389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea530-79b8-498c-9834-49de38bfccf5 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.263430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 83/182] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:21.264497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e3d8-d65e-42fa-a81f-3c807ad16013 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.264761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e3d8-d65e-42fa-a81f-3c807ad16013 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.265197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e3d8-d65e-42fa-a81f-3c807ad16013 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.266279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 84/183] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:21.269935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e3d8-d65e-42fa-a81f-3c807ad16013 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.291870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.311066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f0205-bf93-4ab7-bf54-5de118ec4a97 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.344478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.345124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.345431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.345705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.346537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.346846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.348315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.353929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 100/184] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.354684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.381926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.383794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e3d8-d65e-42fa-a81f-3c807ad16013 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.422180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.423060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.423344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.423616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.424976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.429120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.436634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 85/185] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.436634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.484776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6282d-c75f-436a-8d4b-37eac62cc538 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.529093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.530029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.530204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.530465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] PATH_INFO: `/v3/projects/160e05ed19894d30829fc6a0e85de880` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.531674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] RBAC: Authorizing `identity:get_project(project_id=160e05ed19894d30829fc6a0e85de880)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.533125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.534941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 101/186] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/projects/160e05ed19894d30829fc6a0e85de880 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.535716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.549217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.561238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b39ed1-8265-436f-8d96-19738f441526 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.599871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.600664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.600947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.601251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.611629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f4bd6d63-3685-4558-b14d-430f63d266bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:21.619194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:21.620072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:21.621983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 86/187] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:21.622831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.622831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.622831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.633483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.639041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:21.640271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:21.645607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e423d-ff0e-4e58-af67-36e4b6f50399 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.676576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.677466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.677750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.678025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] PATH_INFO: `/v3/users/98f7e7d4e99b4daa83470756a399baa1` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.679212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] RBAC: Authorizing `identity:get_user(user_id=98f7e7d4e99b4daa83470756a399baa1)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.680664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f3b22-4251-48f4-9962-c7eec58156d3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.682321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 102/188] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/users/98f7e7d4e99b4daa83470756a399baa1 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.684070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.684357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.684637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.700460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:21.701657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:21.705962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 87/189] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:21.706793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.767832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 103/190] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:21.768927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.791342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.813061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7741a-e9cf-4544-8267-b6474ea5c85e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.846290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.846951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.847236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.847516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.848352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.848676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.850156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.856447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 88/191] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=0571188a7619422aac6f72fee883d8d6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.857334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.866075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.880114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9f548-4a62-4f24-ab31-60cae18ab79d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.919484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.920156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.920444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.920720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.921603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=160e05ed19894d30829fc6a0e85de880, user.id=fa38071580e54275a3c6d76058090878) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.921926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.923408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.930741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 104/192] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/role_assignments?user.id=fa38071580e54275a3c6d76058090878&scope.project.id=160e05ed19894d30829fc6a0e85de880&role.id=0571188a7619422aac6f72fee883d8d6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.931916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.006491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68e43e-052e-4097-a55d-9481bfe14bc4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.068261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.068880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.069188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.069485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.070255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:22.070567 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.071231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a38e6-3484-4a14-948a-1f17e4ce9f6d None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.071971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.082649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 89/193] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.083508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.092402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.123544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.124199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.124486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.124770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.150327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.200960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.202458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.203168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dfc23-fe95-4107-9db3-59ebb3df1589 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.236961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.237935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.238547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.238547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.248491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:22.256534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:22.257493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 90/194] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:22.258805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.307057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 105/195] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:22.307888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.322288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.359163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.359353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c9c30-416c-4f79-9642-7312384e0439 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.392747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.393584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.393870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.394145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.395328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.396834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.398638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 91/196] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.399245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.405774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86f99-b772-4db0-babf-502f9a3f4185 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.439196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.440156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.440445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.440726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.456358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.457870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.459587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 106/197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.460353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.509049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4170c2-a07e-481e-88a5-3e637d6f1798 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.538976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.543725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.549984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.555209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 92/198] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.556076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.558426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199035ab-4e69-4d82-ac83-eb0a13e5b04e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.569282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.592945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.593775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.594056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.594333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.595525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.597543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.599513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 107/199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.600586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.661636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfe63e-898d-4c9a-9202-61a74769a41a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.695074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172b69-1a2f-40ef-b8bf-cafeb5cd6e59 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.696681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.697382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.697668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.697948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.698840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.700789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.728048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.728684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.728965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.729269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.774848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.784734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 93/200] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:22 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:22.791688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.805509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.807073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.847035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.903692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4701b-e69c-42a0-b2c4-fe19d6905328 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.916536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 108/201] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:22.918122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.946407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.947297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.947583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.947863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.950047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.951503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.953210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 94/202] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.953920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.014197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.041732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af38c1-13ab-4f70-8774-7c925dd1f4c6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.079287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d3cc6-1571-41de-ac00-116f3333fba0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.090234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.090894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.091177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.091455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.092296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2, user.id=98f7e7d4e99b4daa83470756a399baa1) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.092609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.094640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.100576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.102340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 109/203] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/role_assignments?user.id=98f7e7d4e99b4daa83470756a399baa1&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=8185353471b64ed18c802551c3ad37a0 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.102986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.124145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.125040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.125360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.125635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.128002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.130492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.134607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 95/204] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.141506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.242779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c53d4-424c-4f78-bac8-6b56c4efb9a2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.282431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.283455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.284034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.284507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.294496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54267b46-d075-4379-ae25-9fbdb28b488c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.314530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.336936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.337801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.338232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.338624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.339617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=30ea3a22d4f9482197cef733ab2cfe16) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.340034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.341625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.348100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 96/205] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/role_assignments?user.id=30ea3a22d4f9482197cef733ab2cfe16&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.391070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.392499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.505143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 110/206] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:23.512040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.576704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.637356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce89e3-81ce-474a-8458-51932bf5a3f6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.679667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.680459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.680738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.681015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.682250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.683735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd5ee-b6eb-4830-a76e-6f4760803322 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.685925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 97/207] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.691624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.742221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.799043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caa289-2645-4ad0-95dd-90d06d8b9fc8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.834260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.834923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.835208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.835485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.836325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=30ea3a22d4f9482197cef733ab2cfe16) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.836640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.838130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38747a10-aa4e-46d6-aed5-13de8d6d90b3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.844647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 111/208] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/role_assignments?user.id=30ea3a22d4f9482197cef733ab2cfe16&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.036532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f8f8-0e1e-401c-96ca-849cdf5743c8 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.037054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f8f8-0e1e-401c-96ca-849cdf5743c8 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.037178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f8f8-0e1e-401c-96ca-849cdf5743c8 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.038649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 98/209] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:25.044243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ade2b-23a0-48bf-82f4-dc9b5a0af2b9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.044631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ade2b-23a0-48bf-82f4-dc9b5a0af2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.044801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ade2b-23a0-48bf-82f4-dc9b5a0af2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.062340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-274ade2b-23a0-48bf-82f4-dc9b5a0af2b9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:25.063511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-274ade2b-23a0-48bf-82f4-dc9b5a0af2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:25.133545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 112/210] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:25.158626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6f8f8-0e1e-401c-96ca-849cdf5743c8 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.168464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5703e5-fabd-41a6-b9c6-c1501d1f57bb None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.168755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5703e5-fabd-41a6-b9c6-c1501d1f57bb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.169038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5703e5-fabd-41a6-b9c6-c1501d1f57bb None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.170468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 113/211] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:25.175375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.175669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.175950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.196410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:25.197640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:25.238323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.278442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6f8f8-0e1e-401c-96ca-849cdf5743c8 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.288680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 114/212] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:25.305545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.312809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.318191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.319055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.319338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.321292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.331947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:25.340837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:25.341762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:25.342561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 99/213] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:25.347949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.417735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9073a1f-08b2-4bdb-8584-f74beb2d59c3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.456371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.457196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.457474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.457749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.458945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.459140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605a4f-9a51-44a6-8ef9-3ae91c5324be None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.460417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dd777-ca96-4e7a-8a17-60982c55aa36 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.465507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 115/214] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.465507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905f05-4e36-45ea-8cf1-9407c6b2f2c3 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.465507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905f05-4e36-45ea-8cf1-9407c6b2f2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.465507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905f05-4e36-45ea-8cf1-9407c6b2f2c3 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.465840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 116/215] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:25.469292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4905f05-4e36-45ea-8cf1-9407c6b2f2c3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.503717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.504494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.504778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.505052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.505831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:25.506136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.507517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5238c7-4c98-4344-b503-47e3275143f2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.508934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.517674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 100/216] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:25.518911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b5d2-be23-4705-a469-45400567d461 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.527478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b5d2-be23-4705-a469-45400567d461 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.527478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b5d2-be23-4705-a469-45400567d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.536621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8303b5d2-be23-4705-a469-45400567d461 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:25.537843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8303b5d2-be23-4705-a469-45400567d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:25.572388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.586225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4905f05-4e36-45ea-8cf1-9407c6b2f2c3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.617213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 101/217] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:25.617213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b5d2-be23-4705-a469-45400567d461 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.636050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.637309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.637591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.638116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.638940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.640500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.708102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b5d2-be23-4705-a469-45400567d461 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.733690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 117/218] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:25.734881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.751739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.752650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.752934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.753241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] PATH_INFO: `/v3/users/6757aee315664d83b496a40230a77834` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.770525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] RBAC: Authorizing `identity:get_user(user_id=6757aee315664d83b496a40230a77834)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.772095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.778236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 102/219] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/users/6757aee315664d83b496a40230a77834 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.779983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.813468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.878059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.892163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d36f8-30bd-4207-9796-49999d4fcfe5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.898091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b15324-4caa-4b4f-a7c0-add07891b76c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.936524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.937443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.937730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.938009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.942052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.942700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.942980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.943363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.951492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d92d8a65-e9d3-4afc-8eb1-814fb9fe0317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:25.960322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:25.961293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:25.962096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 118/220] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:25.962819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb399f-ec34-4511-bb09-94ed4603e6e9 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.963118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb399f-ec34-4511-bb09-94ed4603e6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.963454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb399f-ec34-4511-bb09-94ed4603e6e9 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.969092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 119/221] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:25.970217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb399f-ec34-4511-bb09-94ed4603e6e9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.029003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.030583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c7c73-7556-48d2-b4e9-e2efe7ed25cc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.067839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.112903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb399f-ec34-4511-bb09-94ed4603e6e9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.149189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.149793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.150075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.150348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.151100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.151403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.152427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 103/222] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:26.152819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.153747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.154140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.154423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.166023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 120/223] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:26.167080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.172008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:26.173478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:26.187760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.247370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 104/224] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:26.248500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.274302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ccfeb-28c4-4659-8f78-d55dbe3729da None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.320097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.321572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.322291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.323175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.325169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.325604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.328349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.331124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 121/225] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.332139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.382119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b90cc9-765e-47bd-97df-c5fe234f6bab None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.428845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.443610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.445190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.445675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.446456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.448628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.450899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.451694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce08b-d97a-4a3c-bf53-9eed6cc25062 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.453559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 105/226] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.454944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.528450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.529291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.529578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.529856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.543713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:26.554484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:26.555384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:26.556179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 122/227] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:26.556841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.617793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b54a9-a3d3-4574-992f-71543a6fad59 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.653433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.663845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.665040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.665517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.668361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.668361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=6757aee315664d83b496a40230a77834) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.669061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.671424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.679457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 106/228] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/role_assignments?user.id=6757aee315664d83b496a40230a77834&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=0571188a7619422aac6f72fee883d8d6 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.680552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.703256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b19ea-ecf8-4d13-b1e2-58828b58d866 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.762543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.764077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.764544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.765023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.766486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.768118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.770377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 123/229] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.779315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.797392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.861665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fdd51-eca8-4c25-8ef4-59f962734647 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.909421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.910483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.910968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.911468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.912415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.912922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.914545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.923928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 107/230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.930182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.941847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a17f2-a568-4837-a260-ea17d58f2815 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.984502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.985422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.985708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.985984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/30ea3a22d4f9482197cef733ab2cfe16/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.990127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, user_id=30ea3a22d4f9482197cef733ab2cfe16, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.991590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.015469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.066937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b7b0d-0df9-4519-aeb8-815e94f71116 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.076612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 124/231] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:26 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/30ea3a22d4f9482197cef733ab2cfe16/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:27.118459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.119371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.119660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.119940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.121538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.123224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.125031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 108/232] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:27.131074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.235498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.280812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b51c6d-7596-4e18-88cd-7166730196b1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.318797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.319652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.319938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.320217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.321517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.324444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518daf05-b65f-41f5-b142-fe3693a2871a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.327633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 125/233] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:27.333522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.453398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05566-cd61-458d-9048-68c4b762e92b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.478981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.499478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.500294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.500575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.500851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/30ea3a22d4f9482197cef733ab2cfe16/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.507070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, user_id=30ea3a22d4f9482197cef733ab2cfe16, role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.509129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c4fa-032d-4b0a-9171-63bbe751870b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.623932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 109/234] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:27 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/30ea3a22d4f9482197cef733ab2cfe16/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:28.069439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7a53-bc84-462e-ba1b-f7de717fec3a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.275535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7a53-bc84-462e-ba1b-f7de717fec3a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.275535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb7a53-bc84-462e-ba1b-f7de717fec3a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.275535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 126/235] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:28.276243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.276243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.276243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.276243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:28.276243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d38d7-45ab-4629-8ca4-44900109dc8e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d38d7-45ab-4629-8ca4-44900109dc8e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d38d7-45ab-4629-8ca4-44900109dc8e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 127/236] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.277447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.277826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.277826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:28.277826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:28.278023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 110/237] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:28.278023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.323103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 128/238] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:28.362103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.402612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500897d1-56e7-4afc-ab26-17d6d48af4e9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.423966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.469534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.470067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.470726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.470930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.471168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.472520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.476281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.478009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 111/239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:28.483465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.535574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a2a75-78e9-4ac6-a861-3ff42a290e6a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.597091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c36d2-99c8-4014-a9e1-10f6885354b6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.607425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.608246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.608895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.608895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.634730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.635764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.636819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:28.636939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.637467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.638635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.640327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:28.658700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:28.663278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:28.664138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 129/240] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:28.665842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.737107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 112/241] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:28 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:28.774705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed5c22-360e-4c92-9441-b49884229b24 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.802472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.811929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.816831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.825792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 130/242] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:28.831236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.869694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.945191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de6ece-6317-443d-a415-e7ab2662ba71 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.980797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.981736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.981991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.982266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] PATH_INFO: `/v3/users/ed80c1a9d7944c038368c5e7b65c27c8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.996711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] RBAC: Authorizing `identity:get_user(user_id=ed80c1a9d7944c038368c5e7b65c27c8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.998278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.000526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 113/243] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/users/ed80c1a9d7944c038368c5e7b65c27c8 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:29.005947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.108644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83202ca3-91af-4436-a16c-4c4df4d3835f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.143836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.144401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.144683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.144954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.211664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.215102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.216672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb53f3-7765-4e5a-8cf5-64db846b43aa None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.322378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 131/244] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:29.327775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.381490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b6b6-c68b-4829-a92c-454648918f4e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.381823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b6b6-c68b-4829-a92c-454648918f4e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.382062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b6b6-c68b-4829-a92c-454648918f4e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.384728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 132/245] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:29.389078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.389387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.389678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.412425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:29.413595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:29.428646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.439805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579dee3-5bb8-412d-87ef-6f869e47e5c1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.484687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.485747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.486351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.486691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.488944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.491106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.493889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 114/246] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:29.496895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 133/247] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:29.499917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.520885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.561069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.617278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237511-950f-4a36-8e45-539e2c14d391 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.655364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a52e8-20a4-4165-b2be-88b2e678d9de None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.662981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.663577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.663858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.664135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.664975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=ed80c1a9d7944c038368c5e7b65c27c8) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:29.665318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.666757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18508c-21f9-4651-82dd-368507829e2a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.673393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 115/248] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/role_assignments?user.id=ed80c1a9d7944c038368c5e7b65c27c8&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=8185353471b64ed18c802551c3ad37a0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:29.677280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1050a-9f61-4156-a2d8-4cc92f5ca900 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.677869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1050a-9f61-4156-a2d8-4cc92f5ca900 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.678356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1050a-9f61-4156-a2d8-4cc92f5ca900 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.680051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 116/249] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:29.685974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.686645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.687125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.704288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.705117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.705457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.705741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.711423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:29.713018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:29.730709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:29.735420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:29.740320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:29.741337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:29.742165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:29.750028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 134/250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:29.756020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.799896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 117/251] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:29.832639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.911971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.953649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b8313-c44c-4d66-8173-cde35cdab751 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.996489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.997186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.997492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.997776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.998540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:29.998855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.000781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.010201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 135/252] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:30.016815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.055103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c3d9-656e-4f45-882a-7e185760d525 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.127900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e697a-d4da-4ab4-b2ab-4b7fc7bed19a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.158417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.159226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.159508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.159786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.164249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.165118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.165541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.165821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] PATH_INFO: `/v3/users/6757aee315664d83b496a40230a77834` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.167270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] RBAC: Authorizing `identity:get_user(user_id=6757aee315664d83b496a40230a77834)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.168952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.170996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 136/253] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/users/6757aee315664d83b496a40230a77834 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:30.176341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.182439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:30.194223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:30.195152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:30.195955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:30.202420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 118/254] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:30.203062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.208225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.234636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.282891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083ce6-bb79-485b-a34c-4aca264594a8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.319774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.320652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.320943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.321253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.322711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.324749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.326909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 137/255] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:30.332390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.370277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f3d97-28b9-4115-980f-5c49c79c2a8c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.416624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.450375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.451366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.451996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.453704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.454659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:30.455868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.458119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ca4a4-d039-4534-a46e-dbf1f5d23a51 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.470915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 119/256] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:30.474020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8fc70-9b7f-4d8f-8d5c-fe0be9e06e22 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.477376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2c5f8-5c6e-430d-81db-26ce7190f53c None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.478633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2c5f8-5c6e-430d-81db-26ce7190f53c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.479092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2c5f8-5c6e-430d-81db-26ce7190f53c None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.489086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 120/257] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:30.490192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2c5f8-5c6e-430d-81db-26ce7190f53c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.539711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.541771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.543733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.544220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/6757aee315664d83b496a40230a77834/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.548242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, user_id=6757aee315664d83b496a40230a77834, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.550888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418fed6-036f-4861-8e4b-a7c20273fb1e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.609756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.638468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2c5f8-5c6e-430d-81db-26ce7190f53c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.656024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 138/258] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:30 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/6757aee315664d83b496a40230a77834/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:30.657399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a61864-449e-445b-a6d6-87335a19663f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.657706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a61864-449e-445b-a6d6-87335a19663f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.661285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a61864-449e-445b-a6d6-87335a19663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.692182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-68a61864-449e-445b-a6d6-87335a19663f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:30.694134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-68a61864-449e-445b-a6d6-87335a19663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:30.704738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.705688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.705986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.706300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.709923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.711419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.713198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 121/259] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:30.718660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.841033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 139/260] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:30.874642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127d88-8c9a-4567-8ee5-12e08feb862b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.912352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a61864-449e-445b-a6d6-87335a19663f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.930733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.931172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.931469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.931724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.963291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.027527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.032307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.034617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.084194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a61864-449e-445b-a6d6-87335a19663f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.140021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.141013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.141330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.141611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.166190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:31.174886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:31.175789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:31.176608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:31.181648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 140/261] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:31.188352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.280502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 122/262] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:20:31.285631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.421682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.422273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23a764-4dbf-4ad3-8f57-e594547e1464 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.457601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.468624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.469628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.470132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.470613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.471579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.472087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.473720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.493773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 141/263] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.499840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.546481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccf7d5-8a3a-4f8a-b571-a689b7a84a52 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.609552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.611820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.612990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.614050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.615626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.619667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.624095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 123/264] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.629391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.692834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db687cab-da9b-458b-b17e-734446d3a451 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.771866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.772751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.773054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.773598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] PATH_INFO: `/v3/users/30ea3a22d4f9482197cef733ab2cfe16` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.778167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] RBAC: Authorizing `identity:get_user(user_id=30ea3a22d4f9482197cef733ab2cfe16)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.779948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.782250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 142/265] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/users/30ea3a22d4f9482197cef733ab2cfe16 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:31.788835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.856185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.858588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d716e2-cc38-4dea-affe-afec2982e5a3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.893562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.935452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.936130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.936417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.936700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.937586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=30ea3a22d4f9482197cef733ab2cfe16) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.937904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.939609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.954180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 124/266] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/role_assignments?user.id=30ea3a22d4f9482197cef733ab2cfe16&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.975633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69847b-955d-4aab-a368-96ee01c6c2c2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.036555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.037241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.037527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.037801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.132789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.134334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597de811-a1ca-44ea-a075-d50996c68c0c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.217889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.283187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 143/267] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:20:32.288568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.420459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42ea9-4f04-434c-8574-bdc8b4d95ab3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.475198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.476792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.481337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.481828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.483413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.484655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad3264-9233-4157-a3ca-84ab04a0bb08 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.485094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad3264-9233-4157-a3ca-84ab04a0bb08 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.485280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad3264-9233-4157-a3ca-84ab04a0bb08 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.485901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.486875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 144/268] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:32.488062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 125/269] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:32.494447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378cab-d1e8-4208-bf43-65bf0d1c5d9e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.495046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378cab-d1e8-4208-bf43-65bf0d1c5d9e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.496594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.499704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378cab-d1e8-4208-bf43-65bf0d1c5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.521595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bd378cab-d1e8-4208-bf43-65bf0d1c5d9e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:32.522791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bd378cab-d1e8-4208-bf43-65bf0d1c5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:32.562672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.629866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 145/270] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:32.631185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604cfa-1ae2-4ff3-ba4d-122f2fb7fdb0 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.631912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604cfa-1ae2-4ff3-ba4d-122f2fb7fdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.632192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604cfa-1ae2-4ff3-ba4d-122f2fb7fdb0 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.633553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 146/271] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:32.634722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9cf71-330c-46b9-ad6d-cc0079be25d7 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.635007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9cf71-330c-46b9-ad6d-cc0079be25d7 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.635279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9cf71-330c-46b9-ad6d-cc0079be25d7 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.641341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 147/272] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:32.643629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88336033-0468-45c1-b2ce-b76895988216 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.644032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88336033-0468-45c1-b2ce-b76895988216 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.644624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88336033-0468-45c1-b2ce-b76895988216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.649594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be4487-b5e3-492b-b7a3-7026cc12b726 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.665392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-88336033-0468-45c1-b2ce-b76895988216 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:32.666744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-88336033-0468-45c1-b2ce-b76895988216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:32.683791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.684455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.684740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.685020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.685879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=30ea3a22d4f9482197cef733ab2cfe16) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:32.686194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.687634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c77929-0b6b-44ba-93cb-80de426a9f72 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.693909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 126/273] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/role_assignments?user.id=30ea3a22d4f9482197cef733ab2cfe16&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:32.696761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.696761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.696761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.722322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:32.724588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:32.750769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 148/274] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:32.751804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88336033-0468-45c1-b2ce-b76895988216 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.774874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.820240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 127/275] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:32.825580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.890650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.909642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88336033-0468-45c1-b2ce-b76895988216 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.959947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.961193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.961193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.961360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.975268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:32.975463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012a0a-4443-4a46-b71f-546b174c8316 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:32.983856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:32.984889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:32.985878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 149/276] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:32.986748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.019166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.020289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.020561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.020838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.022066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.023523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.025211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 128/277] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.025974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.075056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.098249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de641-cdf3-4a74-aed6-0cddf9f25592 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.131733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc4c8-5f5c-4570-9814-24b17b4f908b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.136891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.138054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.138342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.138619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.139844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.149651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:33.158070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:33.159177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 150/278] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:33.160176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.171043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.171715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.171998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.172274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.173309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:33.173947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.175861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.185807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 129/279] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.186814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.272089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b282d0f-a461-4342-a86a-eec2f5907f39 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.303108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb00a6-13e0-452b-83b7-9202d65a206c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.316343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.317269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.317555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.317843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.318556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.320007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.328895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.340319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.340991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.341355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.341574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.342360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:33.342670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.344036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.352043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 130/280] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.352892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.383824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.437382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 151/281] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:33 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:33.450285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.456029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc313c6-0b1c-45c2-8201-472453397226 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.496849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.497777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.498196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.498469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] PATH_INFO: `/v3/users/49f6b8cab0f448e3981757f23a3e3cc6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.514789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] RBAC: Authorizing `identity:get_user(user_id=49f6b8cab0f448e3981757f23a3e3cc6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.516292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd6a72-644d-47e0-9fdb-6a995e3c87f9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.518133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 131/282] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/users/49f6b8cab0f448e3981757f23a3e3cc6 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.519653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdc2d7-dcef-40c0-aa42-3aeac2eb54f5 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.519949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdc2d7-dcef-40c0-aa42-3aeac2eb54f5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.520231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdc2d7-dcef-40c0-aa42-3aeac2eb54f5 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.521790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 132/283] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:33.526718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdc2d7-dcef-40c0-aa42-3aeac2eb54f5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.574532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3491-bb44-4ac0-b8e3-117e3cc289f7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.616486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.618724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.618724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.618724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] PATH_INFO: `/v3/users/ed80c1a9d7944c038368c5e7b65c27c8` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.619237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] RBAC: Authorizing `identity:get_user(user_id=ed80c1a9d7944c038368c5e7b65c27c8)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.621313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708fdf-31f4-4e06-8cc1-6eb1ac20dc4f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.623032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 152/284] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/users/ed80c1a9d7944c038368c5e7b65c27c8 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.624276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.624561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.624837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.635211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.642208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:33.652757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:33.652903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdc2d7-dcef-40c0-aa42-3aeac2eb54f5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.689978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.690648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.690932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.691207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.692300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, user.id=49f6b8cab0f448e3981757f23a3e3cc6, scope.system=all) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:33.692748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.694276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.700397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 133/285] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/role_assignments?user.id=49f6b8cab0f448e3981757f23a3e3cc6&scope.system=all&role.id=0571188a7619422aac6f72fee883d8d6 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.705297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.719880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.737248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 153/286] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:33.763172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.836475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73396a14-0e51-48ad-b062-782c4292786b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.875849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:33.876710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.876991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.877292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.878497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:33.879957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:33.881713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 134/287] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.890064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.912916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1eaaa-a3a3-4766-ab4a-98b35b7befd7 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:33.937559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.005691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd524c4-ad2b-42b2-9fef-bf835347b89a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.014459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.015269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.015550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.015824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.037855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:34.050896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:34.051846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:34.052883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 154/288] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:34.062032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.069841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.070736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.071015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.071289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/ed80c1a9d7944c038368c5e7b65c27c8/roles/8185353471b64ed18c802551c3ad37a0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.077775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, user_id=ed80c1a9d7944c038368c5e7b65c27c8, role_id=8185353471b64ed18c802551c3ad37a0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.079231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.130797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.184376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 135/289] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:33 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/users/ed80c1a9d7944c038368c5e7b65c27c8/roles/8185353471b64ed18c802551c3ad37a0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:34.199084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51503-341c-459f-940e-0eada7d1a089 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.244260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.244946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.245289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.245571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.246390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.246786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.248308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.264059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.265127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 155/290] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:34.409272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.425831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb694a9-1598-4d9d-9260-0e0d87caf3a3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.465444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.466339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.466620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.466897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] PATH_INFO: `/v3/users/6757aee315664d83b496a40230a77834` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.468130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] RBAC: Authorizing `identity:get_user(user_id=6757aee315664d83b496a40230a77834)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.469603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.471490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 136/291] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/users/6757aee315664d83b496a40230a77834 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:34.476891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.598298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191277ae-163d-4aa6-aa99-fb9ec1d63db4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.611654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.645889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.646568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.646858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.647140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.717787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.719307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.829289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 156/292] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:34.834898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.957785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe38a-df20-480f-a528-c70335929997 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.001020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.001834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.002115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.002389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.005887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.007327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc72cd4-2f03-4323-945b-a3c13ec55917 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.009362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 137/293] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:35.016601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:35.041205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:35.178578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d7020-57e7-4cf9-9f40-65430d7d8a99 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.237875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.238550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.238836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.239113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.239963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=6757aee315664d83b496a40230a77834) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:35.240276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.241784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee1b56-7c26-4e6d-b320-77ab18d176dd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.254009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 157/294] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:35 2022] GET /identity/v3/role_assignments?user.id=6757aee315664d83b496a40230a77834&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=0571188a7619422aac6f72fee883d8d6 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:36.021844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce187cd-25a5-417c-bf85-0bb069cc381b None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.022393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce187cd-25a5-417c-bf85-0bb069cc381b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.022393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce187cd-25a5-417c-bf85-0bb069cc381b None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.024001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 138/295] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:36.029312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.030167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.030656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.048613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:36.050182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:36.107052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228d23-6c38-4b11-876d-57fd245291ac None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.107687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228d23-6c38-4b11-876d-57fd245291ac None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.108191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228d23-6c38-4b11-876d-57fd245291ac None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.110007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 139/296] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:36.115359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48ecb9-f03b-42ce-ac07-afc7de358352 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.115872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48ecb9-f03b-42ce-ac07-afc7de358352 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.116365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48ecb9-f03b-42ce-ac07-afc7de358352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.129181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 158/297] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:36.134377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4d48ecb9-f03b-42ce-ac07-afc7de358352 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:36.135562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4d48ecb9-f03b-42ce-ac07-afc7de358352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:36.146875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.149947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.156673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.217644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 140/298] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:36.217644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9335c7-55fe-49c4-8e8d-f63c68ead017 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.217644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9335c7-55fe-49c4-8e8d-f63c68ead017 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.217992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9335c7-55fe-49c4-8e8d-f63c68ead017 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.223223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 141/299] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:36.224564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.225020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.225475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.245185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:36.246597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:36.254879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030448d5-c8b7-4c6c-9a79-43119723a553 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.290571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.291281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.291567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.291841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.293625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:36.295065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:36.296856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 159/300] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:36.301844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.318942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 142/301] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:36.320000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.398825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.410976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.423779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8e31f-6324-46cc-8bde-3d128bf26785 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.437690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdaea0-0b94-47eb-8fd7-6c78bdfeb298 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.462737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.463530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.464430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.464706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.476320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:36.477918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.478775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.479198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.480195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] PATH_INFO: `/v3/groups` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.481013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:36.482696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:36.484595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:36.485681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 160/302] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:36.486947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.567175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 143/303] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:36.568132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.596175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3080-dc64-402b-b2db-9ff815730d46 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.633406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.634250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.634541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.634814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.638197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.648371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ffef768e-69e7-42b0-af60-f0924ec6a5f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:36.656923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:36.658387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:36.659240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 161/304] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:36.659971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaa7f9-9791-40c4-8b1a-aecc93053828 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.659971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaa7f9-9791-40c4-8b1a-aecc93053828 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.660125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaa7f9-9791-40c4-8b1a-aecc93053828 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.661454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 162/305] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:36.664552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaa7f9-9791-40c4-8b1a-aecc93053828 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.703161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eaeba-61e2-4eb4-9cb0-2581db10894d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.742402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.743362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.743809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.744633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.745810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:36.746136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:36.747523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1901e1-1b6f-4357-b2ca-02b8c741fe6c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:36.750761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.759941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 144/306] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:36.762387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.762961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.763303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.781150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:36.781499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaa7f9-9791-40c4-8b1a-aecc93053828 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.782425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:36.819362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.820577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.821947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.822438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.823920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:36.824422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:36.826471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:36.837881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 163/307] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:36.840011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.856013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 145/308] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:36.856702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.910615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.957974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d99b7-2689-42ef-a3f7-a553d6316d73 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.961561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633d3ef-2a7c-42c5-aace-97bf2c1acd38 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.988209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.992886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.993793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.994083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.994363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] PATH_INFO: `/v3/users/2becf226dea44f6396e956fd3bd649a2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.997621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.999254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.999537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.999811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] PATH_INFO: `/v3/users/49f6b8cab0f448e3981757f23a3e3cc6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.001004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] RBAC: Authorizing `identity:get_user(user_id=49f6b8cab0f448e3981757f23a3e3cc6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.002465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.004691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 146/309] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/users/49f6b8cab0f448e3981757f23a3e3cc6 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.005319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.014328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] RBAC: Authorizing `identity:get_user(user_id=2becf226dea44f6396e956fd3bd649a2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.015789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.022196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 164/310] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/users/2becf226dea44f6396e956fd3bd649a2 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.022196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.121850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c6f95-ba50-49c7-8e2e-70184e3321bc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.127503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cda64f-9048-46a2-bbc9-c6955a34a4df None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.143019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.157934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.158782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.159068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.159348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.165896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.166886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.167177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.167452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] PATH_INFO: `/v3/system/users/49f6b8cab0f448e3981757f23a3e3cc6/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.168928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49f6b8cab0f448e3981757f23a3e3cc6, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.169987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:37.170426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:37.177774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:37.178738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:37.179562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 147/311] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:37.184964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.242051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.268574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 165/312] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Jul 12 14:20:37 2022] PUT /identity/v3/system/users/49f6b8cab0f448e3981757f23a3e3cc6/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:37.278504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.299069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b227-6099-4d13-a73d-a9acf4a8a9b9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.348067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.349056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.349469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.349827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.351147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, user.id=2becf226dea44f6396e956fd3bd649a2, scope.system=all) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:37.351527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.353059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.365354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 148/313] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/role_assignments?user.id=2becf226dea44f6396e956fd3bd649a2&scope.system=all&role.id=8185353471b64ed18c802551c3ad37a0 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.427195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb401d8-db36-483c-92dd-3908ef712d1b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.472588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.472877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.473194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.473985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:37.474294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.475913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.486681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 166/314] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.492469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.601677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7677b7b-44e6-408d-88d3-004d86ecc5dd None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.622928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.637457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] PATH_INFO: `/v3/users/ed80c1a9d7944c038368c5e7b65c27c8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] RBAC: Authorizing `identity:get_user(user_id=ed80c1a9d7944c038368c5e7b65c27c8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.646411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 149/315] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/users/ed80c1a9d7944c038368c5e7b65c27c8 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.650049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.696723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.759407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d0e3c-25c7-497b-89b9-c3a4964e6881 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.798866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.799742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.800127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.800511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.873674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.875367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.918005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.976501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 167/316] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:37.982581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:38.083256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cf02e-e47c-4a0d-8887-a6b67e8c4c35 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:38.120641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:38.121804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:38.122265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:38.123132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:38.124563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:38.126178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096137-d726-4dc8-b1b2-43690efa6b50 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:38.128341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 150/317] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:38.136279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:38.248174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf14a35-1abd-406b-b4f2-e8a6edc07597 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:38.350476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:38.351178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:38.351467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:38.351740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:38.352581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0, user.id=ed80c1a9d7944c038368c5e7b65c27c8) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:38.352898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:38.358610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60689705-0edd-4f78-bfcf-3391a7aee976 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:38.379204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 168/318] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:38 2022] GET /identity/v3/role_assignments?user.id=ed80c1a9d7944c038368c5e7b65c27c8&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=8185353471b64ed18c802551c3ad37a0 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:38.643274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47433b-5890-4935-815b-8cb397ee2365 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:38.643605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47433b-5890-4935-815b-8cb397ee2365 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:38.643778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47433b-5890-4935-815b-8cb397ee2365 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:38.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 151/319] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:38.651045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:38.651377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:38.651664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:38.662057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:38.672860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:38.674662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:38.753891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 169/320] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:38.777055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47433b-5890-4935-815b-8cb397ee2365 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:38.832794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:38.877393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47433b-5890-4935-815b-8cb397ee2365 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:38.910554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913668-cbf5-4700-8921-cf715696271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:38.911272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913668-cbf5-4700-8921-cf715696271f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:38.911563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913668-cbf5-4700-8921-cf715696271f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:38.911839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913668-cbf5-4700-8921-cf715696271f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:38.912990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19913668-cbf5-4700-8921-cf715696271f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:38.914453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19913668-cbf5-4700-8921-cf715696271f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:38.916218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 152/321] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:38.922088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.020340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8c26c-5c74-437a-ad6d-ab22a82b0721 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.054195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:39.054856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.055141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.055420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] PATH_INFO: `/v3/groups` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.056103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:39.058241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbbd7e-ad3e-4e6b-a9ec-559da6223fbf None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:39.060975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:39.168992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 170/322] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Jul 12 14:20:38 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:39.431516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae49a-aaef-42b4-aa55-caeed7030fa3 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.431998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae49a-aaef-42b4-aa55-caeed7030fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.431998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ae49a-aaef-42b4-aa55-caeed7030fa3 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.433597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 153/323] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:39.438857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f39f-d6a2-4742-87dc-4b0ae4f522e0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.439457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f39f-d6a2-4742-87dc-4b0ae4f522e0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.439942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79f39f-d6a2-4742-87dc-4b0ae4f522e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.491876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6d79f39f-d6a2-4742-87dc-4b0ae4f522e0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:39.493091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6d79f39f-d6a2-4742-87dc-4b0ae4f522e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:39.564089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 171/324] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:39.588561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ae49a-aaef-42b4-aa55-caeed7030fa3 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.695591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03882ef7-fe32-4b7b-9b76-6a6128ae3263 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.696038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03882ef7-fe32-4b7b-9b76-6a6128ae3263 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.696145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03882ef7-fe32-4b7b-9b76-6a6128ae3263 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.697551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 172/325] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:39.703231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.703517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.703795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.716709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ae49a-aaef-42b4-aa55-caeed7030fa3 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.726562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:39.727751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:39.750804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:39.767997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:39.768715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.769003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.777245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.796715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:39.806542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 173/326] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:39.808090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:39.808971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:39.809807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 154/327] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:39.816909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.838741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.878213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:39.927488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27a59e-77ba-4071-8ebe-dc7852babd5a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.975307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:39.975873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.976158 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.976437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.977276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:39.977540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:39.978945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:39.980695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e76126-e8b8-41fe-a6f0-4e2ef9475fff None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.993188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 174/328] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:40.001306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.081884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.083002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.083462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.083909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.112573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601ebc9-b627-4249-97d6-17fd44563af2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.132247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:40.141371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:40.142296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:40.143103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:40.149275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.151944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 155/329] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:40.153878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.154853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.155142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.155420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] PATH_INFO: `/v3/users/49f6b8cab0f448e3981757f23a3e3cc6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.156663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] RBAC: Authorizing `identity:get_user(user_id=49f6b8cab0f448e3981757f23a3e3cc6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.158077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.161905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.169681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 175/330] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/users/49f6b8cab0f448e3981757f23a3e3cc6 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.170554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.247490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.317734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c97f7-a6a1-4b71-931f-d2b141420378 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.342955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788bbbd-b7b5-4852-b4fc-06b4efe5ff91 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.361195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.361195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.361195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.361195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.361639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] RBAC: Adding query filter params (role.id=0571188a7619422aac6f72fee883d8d6, user.id=49f6b8cab0f448e3981757f23a3e3cc6, scope.system=all) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.361639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.362783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.377323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 176/331] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/role_assignments?user.id=49f6b8cab0f448e3981757f23a3e3cc6&scope.system=all&role.id=0571188a7619422aac6f72fee883d8d6 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:40.413852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.414895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.415361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.415809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.417365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.417972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.419566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.431475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 156/332] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:40.443738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.504512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.575160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dcad0-7f48-4831-b453-866365eb56f5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.615285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.616223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.616510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.616837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] PATH_INFO: `/v3/users/2becf226dea44f6396e956fd3bd649a2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.618104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=2becf226dea44f6396e956fd3bd649a2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.619644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f46139-cfef-4855-b33c-11e149190ca9 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.622265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 177/333] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/users/2becf226dea44f6396e956fd3bd649a2 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:40.627884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.768295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18399ee3-362a-4e36-aaa1-2dad41646c1c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.775942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.812619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.813757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.814180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.814595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] PATH_INFO: `/v3/system/users/2becf226dea44f6396e956fd3bd649a2/roles/8185353471b64ed18c802551c3ad37a0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2becf226dea44f6396e956fd3bd649a2, role_id=8185353471b64ed18c802551c3ad37a0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b2855-0961-4341-be3b-e58c8c15b090 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.886893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 157/334] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Jul 12 14:20:40 2022] PUT /identity/v3/system/users/2becf226dea44f6396e956fd3bd649a2/roles/8185353471b64ed18c802551c3ad37a0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:41.771993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0953d-77bd-4c24-9fa7-c52d7e69a5b9 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.772982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0953d-77bd-4c24-9fa7-c52d7e69a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.773500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0953d-77bd-4c24-9fa7-c52d7e69a5b9 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.775088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 178/335] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:41.780574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.781424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.781930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.800819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:41.802371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:41.881743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 158/336] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:41.938767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0953d-77bd-4c24-9fa7-c52d7e69a5b9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.064993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0953d-77bd-4c24-9fa7-c52d7e69a5b9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.081486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.108693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.109580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.109865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.110142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.126822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fecaff5b-e109-4350-90e7-f5dd3ab617d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:42.134648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:42.136095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:42.136920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 179/337] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:42.143247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.265120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894633-5f02-4622-8380-13cfbbd66cb3 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.265461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894633-5f02-4622-8380-13cfbbd66cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.265699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb894633-5f02-4622-8380-13cfbbd66cb3 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.267082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 180/338] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:42.272990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.276614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1c8c4-9e85-42ce-ba21-cee494ead0bf None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.278264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.278554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.278828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.312802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:42.314206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:42.374530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.375134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.375421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.375700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.376456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:42.376765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.379108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.405507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 159/339] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:42.407331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.466674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 181/340] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:42.498379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.590683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf48b-18f5-45f9-9e5a-a6799b78d770 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.650206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.651589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.652185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.652607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.703345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.720736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.723409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700c0a9-53ef-47ed-913e-e34cc99eb013 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.774341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.777232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216dd50-0b49-4c77-a2b9-96ea8cb4e912 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.796470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.797309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.797592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.797874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.814867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-7e02700e-6b31-49c9-8820-eb4496f41b8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:42.823172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:42.824084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:42.824926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 182/341] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:42.825929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075860-dc73-46d0-8aab-42b69bc39c9f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.826378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075860-dc73-46d0-8aab-42b69bc39c9f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.826803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de075860-dc73-46d0-8aab-42b69bc39c9f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.828214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 183/342] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:42.833078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de075860-dc73-46d0-8aab-42b69bc39c9f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.919696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 160/343] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:42.921021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.921489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.921904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.954270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:42.955478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:42.958749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de075860-dc73-46d0-8aab-42b69bc39c9f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.025485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.026691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.026976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.027252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.028010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.029626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.031029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.039966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 184/344] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.045185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.066244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.075190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.085264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 161/345] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:43.086119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.153882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e8600-93d7-4ef7-8c91-3549571c8862 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.192204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.193142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.193377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.193654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.194864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.196305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61f191-4505-44b0-8e8a-f81f4c86a4d8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.201325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 185/346] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.202762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d6387-b7e7-4065-b7bc-51f140ca761c None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.203057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d6387-b7e7-4065-b7bc-51f140ca761c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.203341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d6387-b7e7-4065-b7bc-51f140ca761c None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.204924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 186/347] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:43.206695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d6387-b7e7-4065-b7bc-51f140ca761c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.219229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76335-7b22-4d6f-a608-7ac0a7f880b6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.255992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.256938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.257421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.257893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.317324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.319173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.328743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.330456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd758-b333-43de-8ecd-188434d90358 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.330960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d6387-b7e7-4065-b7bc-51f140ca761c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.366898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.367526 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.367807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.368082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.381844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:43.393079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:43.394041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:43.394872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:43.397430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 187/348] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:43.398433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.451930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 162/349] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:43.452983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.457566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.458030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.476841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:43.478424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:43.521820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656362fb-b426-4e3b-9067-4b25912472a6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.548895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 163/350] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:43.550082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.574356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.575367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.575890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.576368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.589179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.590971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.592946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 188/351] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.593573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.594188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.615112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.657200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5a5ee-e0d6-4ce8-8238-35cc1df567ee None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.694529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.695244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.695531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.695807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.696547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.697015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.699191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.708868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 164/352] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.709914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.714127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569693d0-78b8-424e-8d6a-29e380fdabc5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.752699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.753949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.754436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.754910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.756348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.758031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.760244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 189/353] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.761793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.815402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.834105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877aa1d-af68-4fe5-aee9-679959d25a54 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.857515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.873489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.874464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.875051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.875681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.876917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4d17-b2c1-42c4-baeb-d87cf00cde5a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.877306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.877779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.879377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.886020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 165/354] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=0571188a7619422aac6f72fee883d8d6 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.887062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.953270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.954420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.954882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.955360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.986320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:43.994596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:43.996136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:43.996949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 190/355] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:43.999389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.063652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123851-6e00-4d4b-b63d-670e4aa2342b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.106384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.107280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.107703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.108139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.131002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.177101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ec38a-f7cf-4f07-81cd-ba4654e0b6c4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.196374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.198068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96afaf-3de4-474c-a5f5-da13c403167a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.218463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.219338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.219623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.219906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.221137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.222752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.224825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 191/356] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.225750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.245340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.332181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 166/357] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:44.333628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e1f0-f7c1-4052-9fb0-dbfe6b5ad897 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.333914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e1f0-f7c1-4052-9fb0-dbfe6b5ad897 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.334194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e1f0-f7c1-4052-9fb0-dbfe6b5ad897 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.335468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 167/358] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:44.337102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b49bd-77eb-48d8-ac0e-6928261dae5a None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.337409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b49bd-77eb-48d8-ac0e-6928261dae5a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.337686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b49bd-77eb-48d8-ac0e-6928261dae5a None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.339048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 168/359] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:44.340345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b49bd-77eb-48d8-ac0e-6928261dae5a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.374765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffb2e8-8a1a-46e6-a268-632325db9bb6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.400809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.419764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.420436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.420723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.421004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.421826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:44.422138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.423696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.437028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 192/360] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.438377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.474561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b49bd-77eb-48d8-ac0e-6928261dae5a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.516106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.516849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.517424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.517424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.518267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:44.518582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.520244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f99125-d68b-42b1-a2f8-3f1a8e554d2e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.530195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.536768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 169/361] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.536768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.536768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.536768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.555050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:44.556279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:44.570806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066abb8-7924-449f-8909-98c48922f6d3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.620667 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.621805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.622096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.622375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.625550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.627383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259af1f-547c-4077-9f13-089392a67c24 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.630241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 193/362] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.632337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.632635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.632919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.658337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 170/363] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:44.659097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.661949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:44.663155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:44.712211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.743228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 194/364] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:44.744210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.792128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771145b-c2a3-44aa-b33e-58d80b6c3341 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.836746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.837466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.837734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.838010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.844837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.883407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60937-5306-42ca-8ff4-086feec4a85d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.916374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.917763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.930519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.932000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.932827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.933420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.934952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.936982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.939579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 195/365] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.940581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.973323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.047868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 171/366] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:45.049251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.069920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2a0-7f81-4016-be99-2397373ed3ed None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.116075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.117120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.117250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.117502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.126454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.130310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:45.139525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.141932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 196/367] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:45.142759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.182648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad91cb-4415-4dad-a06a-7a5f94ebca77 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.220390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.221583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.222101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.222597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.235055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:45.245221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.246193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:45.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:45.250031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 172/368] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:45.251162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.254359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.265903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d9664-4648-4cf2-893b-c76d5e3fd5e2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.305070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.305697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.305987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.306269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.307634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.307954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.309727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.316178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 197/369] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=0571188a7619422aac6f72fee883d8d6 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.316990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.365667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5bc4-fa2e-4d5c-92b3-edd08a8eb539 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.408831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.410233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.410742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.411215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.412617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.414507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.416849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 173/370] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.417982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.426028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.498121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea753712-a2e6-4436-8f00-d565e9247204 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.503678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.537304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a017-f867-4021-b6e3-059d38664f99 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.550592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.551552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.551986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.552407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.553344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.553820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.555518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.566215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 198/371] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.567366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.577181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.578143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.578568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.579067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.580004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.580490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.582234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.592782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 174/372] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.593956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.697861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b2d7a-2ff7-4c70-9346-e232deba328a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.710135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c31527-0ebf-45d8-b381-e050707c17fc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.724425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.737359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.738224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 199/373] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.745905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.749076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.755808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.757064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.757681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.758376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] PATH_INFO: `/v3/users/2becf226dea44f6396e956fd3bd649a2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.760515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] RBAC: Authorizing `identity:get_user(user_id=2becf226dea44f6396e956fd3bd649a2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.762264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.764810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 175/374] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/users/2becf226dea44f6396e956fd3bd649a2 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.765820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.856272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db301a3-7081-40b2-8465-faafbef003dd None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.873866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3baab-262f-400a-ab29-980f42a954d7 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.893785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.894463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.894715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.894988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.912472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.913518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.913997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.914512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.915587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.916095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.918364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.924569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 176/375] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:45.925631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.966822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.978702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.980446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.000805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.044511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1673197-567f-4149-8663-c63441cd8bee None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.085462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.086430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.086881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.087355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.088458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] RBAC: Adding query filter params (role.id=8185353471b64ed18c802551c3ad37a0, user.id=2becf226dea44f6396e956fd3bd649a2, scope.system=all) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:46.088949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.091777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.101189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 200/376] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:46.106011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 177/377] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/role_assignments?user.id=2becf226dea44f6396e956fd3bd649a2&scope.system=all&role.id=8185353471b64ed18c802551c3ad37a0 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:46.107114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.209561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.218143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8ae0c-545a-4dfc-850c-b1ace31c4532 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.253944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.254889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.255176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.255450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] PATH_INFO: `/v3/projects/27c596858f1e4aabb2a1fb3cbe092011` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.256794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] RBAC: Authorizing `identity:get_project(project_id=27c596858f1e4aabb2a1fb3cbe092011)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.258343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.260968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 201/378] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/projects/27c596858f1e4aabb2a1fb3cbe092011 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:46.270271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.403663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb95e-1c80-4acc-8853-38fb54529334 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.428142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.441814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.444254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.444254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.444254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] PATH_INFO: `/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.449992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] RBAC: Authorizing `identity:get_group(group_id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.451456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739195a-7077-41cd-9ca8-2601f009f05b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.453533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 178/379] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:46.459694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.558465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91828164-d846-4429-b859-4885ea8d2882 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.558835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91828164-d846-4429-b859-4885ea8d2882 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.559032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91828164-d846-4429-b859-4885ea8d2882 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.561107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 179/380] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:46.568256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.569751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.570043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.599446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f42b45-9cf4-4d55-95ad-8c10532b80d8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.603245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:46.604998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:46.661885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.662885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.663551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.664044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.665099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] RBAC: Adding query filter params (group.id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7, role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=27c596858f1e4aabb2a1fb3cbe092011) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:46.665650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.668402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.685185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 202/381] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/role_assignments?group.id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7&scope.project.id=27c596858f1e4aabb2a1fb3cbe092011&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:46.717136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 180/382] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:46.741507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.770033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.862678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e22308-7029-4c4a-9308-960401436e24 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.918639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.918639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.918639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.918639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.930294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:46.938736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:46.939710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:46.940560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 203/383] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:46.949211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:47.067779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f07d3a-4d50-4821-a5d5-bd5d1c6e56ef None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:47.086858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:47.105370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:47.109334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:47.117647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 181/384] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:47.123616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:47.235121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:47.252406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4380d34-754e-4f13-9d9c-e6f61ad668bd None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:47.295008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:47.295868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.296154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.296431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.299862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:47.301437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1dc8c-5a0c-4154-a65a-70798e7ac2c7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:47.303456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 204/385] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:47 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:47.309342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:47.426938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9915bd9-c798-4078-9d6d-1e4b44809c79 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:47.445121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101623-560b-4508-afdf-029f15838e7a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.445494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101623-560b-4508-afdf-029f15838e7a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.445808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101623-560b-4508-afdf-029f15838e7a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.447193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 205/386] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:47.459051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.459347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.459628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.483040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:47.484288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.484808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.485404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.486974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:47.488754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:47.490281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:47.491427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 182/387] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:47 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:47.491541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:47.498166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:47.553615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:47.613078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1c846-f1ab-4c60-827c-440bbe50264d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:47.640177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 206/388] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:47.651848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:47.653028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.653546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.654018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/groups/f5f459364e434083927d92ea05e15928/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.656201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, group_id=f5f459364e434083927d92ea05e15928, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:47.657904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:47.681425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:47.753453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 183/389] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Jul 12 14:20:47 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/groups/f5f459364e434083927d92ea05e15928/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:47.784409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:47.805516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da9204-7747-4824-93b2-43f28a71f122 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:47.851240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:47.852136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:47.852420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:47.852693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:47.865806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:47.874829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:47.875782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:47.877045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 207/390] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:47.882595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.017820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f7a6-ef38-4997-85fe-0bc06ecc93c2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.026763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.058488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.059175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.059464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.059745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.060501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:48.060811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.062226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.071982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 184/391] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:48.077198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.207339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d745e-b8c9-417f-94f5-e4bb51121418 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.259371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.260682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.261200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.261683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.263423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.265178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.267351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 208/392] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:48.272605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.408567 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf2fce-ea7f-4787-ac15-ce02a31bc357 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.456541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.457443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.457728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.458003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.459259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.460726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23f457-841e-4397-b416-196ff3f246a3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.462812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 185/393] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:48.469257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.488086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.592874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64841313-b3a7-4c88-baee-aed02afbf4f4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.631757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.631982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66d2e-f50f-4950-a43a-0250e392ee48 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.632262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66d2e-f50f-4950-a43a-0250e392ee48 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.632546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66d2e-f50f-4950-a43a-0250e392ee48 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.632650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.632916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.633231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/groups/f5f459364e434083927d92ea05e15928/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.634400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 186/394] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:48.639084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2, group_id=f5f459364e434083927d92ea05e15928, role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.639851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2bd8-2e44-47a8-975b-1b8b9ef4a29b None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.640141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2bd8-2e44-47a8-975b-1b8b9ef4a29b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.640417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2bd8-2e44-47a8-975b-1b8b9ef4a29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.640521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.668702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-67af2bd8-2e44-47a8-975b-1b8b9ef4a29b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:48.670022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-67af2bd8-2e44-47a8-975b-1b8b9ef4a29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:48.707473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.724965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 209/395] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Jul 12 14:20:48 2022] PUT /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2/groups/f5f459364e434083927d92ea05e15928/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:48.760594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 187/396] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:48.797275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.866059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bb601-3ca2-4c5b-8f48-a94e5d976512 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.866352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bb601-3ca2-4c5b-8f48-a94e5d976512 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.866534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bb601-3ca2-4c5b-8f48-a94e5d976512 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.868170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 188/397] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:48.875113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.875306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.875561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.885938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.921489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:48.922719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:48.923044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c68b3-f02a-42d4-aa37-d345983451e8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.964976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.966304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.966590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.966864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.981482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:48.989274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:48.990185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:48.990992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 210/398] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:48.997277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.046441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 189/399] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:49.072425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.153663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3b1e9-5822-485d-b48f-4b6e84b928b7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.200986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.212024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.212717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.213006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.213330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.214104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:49.214492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.216036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.227302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 211/400] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:49.232916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.249541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.254634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6526bd1-52b5-416e-85ec-a12f67d04eae None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.312069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.312893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.313215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.313718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.329906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:49.340490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:49.341487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:49.342329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 190/401] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:49.348005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.466434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada40622-3be9-4074-994a-4953e1625da9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.508451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.509267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.509628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.509977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.510332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993158-d410-4de3-8c43-a26dc0ffa5c8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.510900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:49.511297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.513280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.522461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 191/402] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:49.528142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.561541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.581514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.582389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.582671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.582945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.584475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.587477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.597181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 212/403] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:49.600551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.707446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a556f7-f6bd-4c6b-804f-40991bfd7da0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.710802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.756516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.757410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.757695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.757974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.759174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.760646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c855e1-9a0d-451e-8d72-1fab55bab222 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.762959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 192/404] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:49.764620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592539-ba75-4e3f-b077-383975832eec None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.764758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fe2c-51fa-4809-9bef-b933993c3e47 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.764992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fe2c-51fa-4809-9bef-b933993c3e47 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.765298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334fe2c-51fa-4809-9bef-b933993c3e47 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.770656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 193/405] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:49.771447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334fe2c-51fa-4809-9bef-b933993c3e47 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.808133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.809084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.809563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.809845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.811852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.813312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c315b-5b87-413f-987f-d140d0248a7a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.815172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 213/406] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:49.816562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.816841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.817114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.843339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:49.844530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:49.878879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.897931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334fe2c-51fa-4809-9bef-b933993c3e47 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.923490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 214/407] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:49.929981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.937487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.939117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.939406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.939684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.940862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.942396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.945686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 194/408] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:49.951031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.021851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.074685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86deea-ad34-4649-896d-aadd47cea047 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.075949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5dc0-4999-43b0-bb5c-78a5b94ca2e3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.111715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.112936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.113068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.113359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/groups/f5f459364e434083927d92ea05e15928/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.115733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, group_id=f5f459364e434083927d92ea05e15928, role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.117289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.123111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.124004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.124290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.124567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/groups/f5f459364e434083927d92ea05e15928/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.126627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] RBAC: Authorizing `identity:create_grant(project_id=ff912b5c50b2457c8c26dbdc6b6834b0, group_id=f5f459364e434083927d92ea05e15928, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.128060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.224892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 195/409] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Jul 12 14:20:49 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/groups/f5f459364e434083927d92ea05e15928/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:50.226213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.241189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 215/410] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Jul 12 14:20:49 2022] PUT /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0/groups/f5f459364e434083927d92ea05e15928/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:50.250884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.368161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc1b1d-7a3f-499d-80bd-fc94f951fd00 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.413388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.414200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.414479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.414753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.434409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ad188975-97a9-42c4-b55b-eaefee51cd1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:50.440289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:50.451714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:50.453053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 196/411] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:50.453830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.558623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91006b-ddd8-41e4-bc9b-cc384a0a602c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.579901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.601909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.602581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.602859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.603131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.604503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:50.604810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.607464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.608347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1358a-6854-48e0-8a31-bd3dc917836f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.608785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1358a-6854-48e0-8a31-bd3dc917836f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.608942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1358a-6854-48e0-8a31-bd3dc917836f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.610704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 197/412] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:50.615954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.616327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.617015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.619267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 216/413] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:50.624969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.635989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:50.637846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:50.758118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 198/414] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:50.771938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904a69f-f20f-4308-ba4f-4fad45b2f4cc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.791192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.799794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.835710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.836572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.836857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.837136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] PATH_INFO: `/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.839787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] RBAC: Authorizing `identity:get_group(group_id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.841280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.844004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 217/415] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:50.853119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.937358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.973094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421962d7-7450-4b5e-9a2f-7d9fad199a58 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.015487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610ccb1-cde6-4714-89fa-231af8832d4c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.020769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.021897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.022411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.022903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.046820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:51.054911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:51.055820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:51.056634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:51.074438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 199/416] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:51.084595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.099986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.101176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.101710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.102236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] PATH_INFO: `/v3/projects/27c596858f1e4aabb2a1fb3cbe092011` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.103786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] RBAC: Authorizing `identity:get_project(project_id=27c596858f1e4aabb2a1fb3cbe092011)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.107039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.111499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 218/417] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/projects/27c596858f1e4aabb2a1fb3cbe092011 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:51.116483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.185570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.214144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067601-42f3-4a05-91b0-4ef487e9582e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.235915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b64545-bdc5-4075-b2f8-8271a41844fc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.254743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.255575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.255960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.256336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.256740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.258683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:51.259194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.261488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5cf3-42f5-47c2-b5bd-047fa1ba37ad None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.271284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 200/418] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:51.272838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f75-d837-412f-9845-ec1fc2167368 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.275197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f75-d837-412f-9845-ec1fc2167368 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.275197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f75-d837-412f-9845-ec1fc2167368 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.275197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 201/419] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:51.277243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1f75-d837-412f-9845-ec1fc2167368 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.282877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.283701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.284056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.284411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] PATH_INFO: `/v3/projects/27c596858f1e4aabb2a1fb3cbe092011/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.288088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=27c596858f1e4aabb2a1fb3cbe092011, group_id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.289926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acb64b-8e5f-4073-a43a-e644ea4691b3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.346860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 219/420] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Jul 12 14:20:51 2022] PUT /identity/v3/projects/27c596858f1e4aabb2a1fb3cbe092011/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:51.348463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.348833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.349219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.368533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:51.370568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:51.395372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1f75-d837-412f-9845-ec1fc2167368 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.425512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.443507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.444509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.444971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.445443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.462065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 220/421] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:51.504024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.576300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.578264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.640024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79425f83-8217-4c65-a82d-9b458501b5d2 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.649192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.680919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.681810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.682605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.682981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.693964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 202/422] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:20:51.703339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:51.703531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:51.712430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:51.713528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:51.714415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 221/423] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:51.721382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.758865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.878961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891edfaa-31ba-48b3-a596-97604266c416 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.901419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f3e7-0b42-4ab8-8915-1d660c90814c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.923926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.924786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.925065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.925359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.927221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.928644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.930520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 203/424] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:51.938044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.966658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.969450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.969739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.970024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.972303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:51.972616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.976669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.993998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 222/425] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:51.999639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:52.058235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.073135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.081207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789c257-f935-4c58-a3d2-24f114527c5b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:52.118732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:52.119650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.120489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.120905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.124148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:52.124600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:52.126312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 204/426] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:52.138853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:52.220963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf224c-da2b-4fb6-86d2-e896dc0b5d50 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:52.298090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:52.298896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.299186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.299463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.342704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0a95-f17f-453d-a7df-7c3173fe2241 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:52.395095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:52.397016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.397325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.397604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.398438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:52.398746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:52.404538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:52.416366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 205/427] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:52 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=0571188a7619422aac6f72fee883d8d6 => generated 611 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:52.440239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:52.441792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cb794-4308-4972-98bd-10c5d0e2a6ca None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:52.487685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.586138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 223/428] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:52.592196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:52.621630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9e1de-48f1-4826-86c8-20ec62996b5a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.621817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9e1de-48f1-4826-86c8-20ec62996b5a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.622090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9e1de-48f1-4826-86c8-20ec62996b5a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.623464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 224/429] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:52.629610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.634758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d44b0-e129-42d4-8207-c1c93a612d23 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.635079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d44b0-e129-42d4-8207-c1c93a612d23 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.635357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d44b0-e129-42d4-8207-c1c93a612d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.671618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-171d44b0-e129-42d4-8207-c1c93a612d23 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:52.672813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-171d44b0-e129-42d4-8207-c1c93a612d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:52.749717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46220-5c21-4833-a431-19ac95b54fca None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:52.768962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 225/430] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 14:20:52.773643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fca5-9586-40b3-b7bc-ac1bb87155f0 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.773928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fca5-9586-40b3-b7bc-ac1bb87155f0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.774202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fca5-9586-40b3-b7bc-ac1bb87155f0 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.776330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 226/431] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:52.781444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.781864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.782219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.800721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:52.801982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:52.808002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:52.808902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.809747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.809747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] PATH_INFO: `/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:52.811091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] RBAC: Authorizing `identity:get_project(project_id=80b95f26e3fe4eb6aea7c1dec2a09eb2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:52.812370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:52.814262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 206/432] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:52 2022] GET /identity/v3/projects/80b95f26e3fe4eb6aea7c1dec2a09eb2 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:52.814946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:52.872280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.890892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 227/433] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:52.890892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:52.933735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:52.945825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c530d-ea57-40fc-ba5b-164912bbf03d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:52.989362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:52.990514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:52.991023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:52.991492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.006570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:53.015670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:53.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:53.017426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:53.018057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb2d3f-cf26-4e9f-bc3f-0972cd0f3aaa None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.019742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 207/434] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:53.020739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.063933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.065211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.065211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.065422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.066614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.068069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.070095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 228/435] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:52 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:53.072123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.144465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.153990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fc80-e106-4ce1-9ac1-d06e02213a7b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.188143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.188935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.189271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.189552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.190570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a09d-fed0-450e-9f2b-b449c64a8eef None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.200612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:53.208248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:53.209190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:53.210018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 208/436] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:53.210694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.228721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.233309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.233986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.234269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.234606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.240100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:53.240100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.240100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.245561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 229/437] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:53.246390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.329370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f8908-d171-4608-bb94-16aa76e0a9d0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.354090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd453-c3bc-4a77-af0d-34ca7c8181bb None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.364998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.365680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.365960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.366236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.367066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:53.367376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.368831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.374114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.382031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 209/438] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=80b95f26e3fe4eb6aea7c1dec2a09eb2&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:53.383470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.397696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.398316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.398601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.398875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.399650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] RBAC: Adding query filter params (name=member) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:53.403456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.404627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.419561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 230/439] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:53.420554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.488616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.514388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce58c30-93d7-4723-b0a6-72745f286d3d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.531149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3e0cf-e220-4f62-acc6-95cfe11cedcf None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.556327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.557566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.557835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.558107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.577363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.578704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.579184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.579837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.648227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.648227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dd0c3-242c-4394-861b-e2e5dcbb334f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.652074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.678477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.680484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef9c51-6cdb-477a-9df3-cf72bc92d5d3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.742476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.767722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 210/440] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:53.769060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae09f7-b846-4935-be35-be8bebc9a1b1 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.769397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae09f7-b846-4935-be35-be8bebc9a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.769679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae09f7-b846-4935-be35-be8bebc9a1b1 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.770943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 211/441] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:53.773117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae09f7-b846-4935-be35-be8bebc9a1b1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.811856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 231/442] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:53.813260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.813539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.813817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.829673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:53.830863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:53.878583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae09f7-b846-4935-be35-be8bebc9a1b1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:53.896658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 232/443] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:53.897335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.911461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:53.912254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.912536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.912808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.914017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:53.915445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:53.921226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 212/444] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:53.921988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:53.925648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:53.967230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.010536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e38ec8-3359-4dc8-9273-f859094947f7 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.019871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757afa36-845b-4a7c-b166-725e28c47158 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.043614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.044462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.044744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.045019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] PATH_INFO: `/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.046210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ff912b5c50b2457c8c26dbdc6b6834b0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.047644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.049362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 233/445] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/projects/ff912b5c50b2457c8c26dbdc6b6834b0 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.053895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.054036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.054680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.054960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.055232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.065986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:54.073593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:54.074546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:54.075359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 213/446] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:54.076051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.131605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.152561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67b37-1349-41b5-90a6-9cfc72f6e0d8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.175232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966acd-5246-4b3b-924b-225f647303d8 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.182115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.186155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.186988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.187261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.187528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.188705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.190160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.191959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 234/447] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.192495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.214616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.215471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.215751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.216025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] PATH_INFO: `/v3/groups/f5f459364e434083927d92ea05e15928` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.217218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] RBAC: Authorizing `identity:get_group(group_id=f5f459364e434083927d92ea05e15928)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.218653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.220440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 214/448] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/groups/f5f459364e434083927d92ea05e15928 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.221011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.292219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2299-59a6-4a5a-b30c-ce48c3ac97ae None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.320525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b874ea-7a32-4456-8085-f96fe0f0b5f1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.324978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.325671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.325954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.326232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.326979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:54.327282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.328659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.336494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 235/449] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.337289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.348032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.354268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.354919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.355199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.356866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.356866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:54.356866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.358083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.363826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 215/450] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.364591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.419812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.454078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf4d1e-fade-447a-9d3a-f94ba5baefcd None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.492984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.493680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.493970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.494332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.495174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] RBAC: Adding query filter params (group.id=f5f459364e434083927d92ea05e15928, role.id=0571188a7619422aac6f72fee883d8d6, scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:54.495489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.496951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.500963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667a5b2-fbdd-461d-93ff-17d84c45bdb2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.503432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 236/451] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/role_assignments?group.id=f5f459364e434083927d92ea05e15928&scope.project.id=ff912b5c50b2457c8c26dbdc6b6834b0&role.id=0571188a7619422aac6f72fee883d8d6 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.550702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.551374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.551678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.551934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.626738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.628226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.669909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.731132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 216/452] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:54.736977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.808479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.837355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7148588-ab78-4c63-9010-80f57522df57 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.871582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.872696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.872975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.873275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] PATH_INFO: `/v3/projects/27c596858f1e4aabb2a1fb3cbe092011` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.874462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] RBAC: Authorizing `identity:get_project(project_id=27c596858f1e4aabb2a1fb3cbe092011)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.875898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.877770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 237/453] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/projects/27c596858f1e4aabb2a1fb3cbe092011 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.883233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.981869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352dada-5862-426d-9ebc-113b46f86380 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:55.015425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:55.016266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:55.016545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:55.016818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] PATH_INFO: `/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:55.017990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] RBAC: Authorizing `identity:get_group(group_id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:55.019417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5cd08-8453-4a00-bc6d-713d77abb6e0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:55.021002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 217/454] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/groups/c8d7ed3cf0aa4d399d3c2bc7a13c1be7 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:55.026944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:55.122908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94c9df-14b4-446e-a520-7cb1d26ce409 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:55.155298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:55.155936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:55.156223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:55.156505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:55.157362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] RBAC: Adding query filter params (group.id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7, role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=27c596858f1e4aabb2a1fb3cbe092011) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:55.157680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:55.159103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64166ff-ef2e-4349-a1ce-e0463e7b7f45 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:55.164705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 238/455] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Jul 12 14:20:55 2022] GET /identity/v3/role_assignments?group.id=c8d7ed3cf0aa4d399d3c2bc7a13c1be7&scope.project.id=27c596858f1e4aabb2a1fb3cbe092011&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:57.763794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb232-7411-49e3-ae59-9956b6788208 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.764274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb232-7411-49e3-ae59-9956b6788208 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.764274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb232-7411-49e3-ae59-9956b6788208 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.765978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 218/456] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:57.770731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6200-6ed5-4d0b-9bc9-931010768038 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.771136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6200-6ed5-4d0b-9bc9-931010768038 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.771321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6200-6ed5-4d0b-9bc9-931010768038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.777489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:57.795147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-129a6200-6ed5-4d0b-9bc9-931010768038 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:57.796393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-129a6200-6ed5-4d0b-9bc9-931010768038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:57.889242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 239/457] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:57.925669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafb232-7411-49e3-ae59-9956b6788208 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:57.934815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:57.935234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8faa5-c12f-49c1-a332-617917409757 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.935523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8faa5-c12f-49c1-a332-617917409757 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.935827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8faa5-c12f-49c1-a332-617917409757 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.939505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 240/458] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:57.942436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5458-c878-45d9-84f0-5b49ecc4bccd None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.942883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5458-c878-45d9-84f0-5b49ecc4bccd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.948325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5458-c878-45d9-84f0-5b49ecc4bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.971155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-27fa5458-c878-45d9-84f0-5b49ecc4bccd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:57.972600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-27fa5458-c878-45d9-84f0-5b49ecc4bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:58.071617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 241/459] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:58.073679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8348f4-bd4f-41a9-bd10-dd398216743e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.074137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8348f4-bd4f-41a9-bd10-dd398216743e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.074586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8348f4-bd4f-41a9-bd10-dd398216743e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.080054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 242/460] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:58.082004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.082441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.082859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.093114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafb232-7411-49e3-ae59-9956b6788208 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.101563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:58.103379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:58.140186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.141302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.141749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.142167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.154254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.155937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.169843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:58.176386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:58.177467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:58.178627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:58.178627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:58.178627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.178627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:58.189176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 219/462] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:58.189176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.189498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 243/462] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:58.190530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.247171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.290359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856ada-b86e-4de4-8ea9-430adae4c9a1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.328951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c09a4-ae67-40e8-8a64-7f15e7189fe5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.329342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.330150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.330692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.330972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.343928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.345413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:58.363094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:58.363971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:58.364866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:58.364866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:58.364866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.364866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:58.364866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 220/463] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:58.365588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.371977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.372645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.372925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.373222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.374050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:58.374362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.376152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa976bb5-6c5c-42b3-b798-87b137f242a7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.389533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 244/464] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:58.391950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f02-92f8-4959-9f7d-ffa2049637ab None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.392444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f02-92f8-4959-9f7d-ffa2049637ab None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.393218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f02-92f8-4959-9f7d-ffa2049637ab None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.395259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 245/465] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:58.396308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35238f02-92f8-4959-9f7d-ffa2049637ab None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.463327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.477275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b0898-9114-4ee2-8b94-063203b57279 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.488169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.511497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35238f02-92f8-4959-9f7d-ffa2049637ab None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.512636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.513472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.513757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.514036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.524955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.527375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:58.545509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:58.546387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:58.547255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:58.547255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:58.547255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:58.547255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:58.547255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 221/466] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:58.547530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.553319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.553983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.554265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.554541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.555277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:58.555564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.556950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f8f1-d969-46e3-99ac-efcabca68c75 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.569084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 246/467] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:58.575266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.575554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.575838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.592696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:58.593831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:58.654043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af0fd8-b0d8-4beb-a703-4e632e17e7c6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.669532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 247/468] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:58.669532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.695849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.696523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.696805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.697123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.697861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.699539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.720535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.772446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3b44c-f115-4c06-8084-76c90cf9bdc0 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.801919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 222/469] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:58 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:58.802803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.819738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.821026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.821350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.821625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.822470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:58.822770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.824639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:58.835977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.837216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 248/470] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:58.843411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:58.931537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dc11e-5bad-45d6-8167-c1df6c6de85f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.963271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4c5d6-3459-4ed1-8211-745bbb7600e7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:58.965995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:58.969146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:58.969846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:58.970298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:58.970472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:58.971180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:58.972623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.000925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.001744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.002026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.002304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.015251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.016699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2623214b-6d9c-4cda-bf38-7bb74ccc65f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:59.033991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:59.034990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:59.035893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:59.035893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:59.035893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:59.035893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:59.035893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 249/471] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:59.036385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0d53f-de3e-4dd2-a3cb-83e3e642938b None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.036652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0d53f-de3e-4dd2-a3cb-83e3e642938b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.036929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0d53f-de3e-4dd2-a3cb-83e3e642938b None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.038488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 250/472] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:59.039176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0d53f-de3e-4dd2-a3cb-83e3e642938b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.079696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 223/473] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:58 2022] POST /identity/v3/users => generated 246 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:59.081108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.167322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0d53f-de3e-4dd2-a3cb-83e3e642938b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.200896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.206230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.206900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.207278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.207554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.208359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.209855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.233967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d707-8b69-404d-beeb-97a72d7ed726 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.270925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.271953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.272446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.272729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.273494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:59.273799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.276276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c58b0-1ee0-4218-a6bb-235ca0d0ae59 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.294577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 224/474] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:59.296238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.296704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.297204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.311904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 251/475] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:59 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:59.313213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.325268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:59.326480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:59.347169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.420137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 225/476] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:59.449941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.497894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a4a8f-84ee-490c-a620-8f5d10ed0b74 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.538773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.539240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.539448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.539736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.540081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.540821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.542369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.586782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550d3a0-43d7-4332-bf3d-8d7154df98e2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.625104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.625814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.626100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.626381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.627143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.628545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.668835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 252/477] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:59 2022] POST /identity/v3/users => generated 249 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:59.695716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.sql.core [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:20:59.695716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 14:20:59.695716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [parameters: {'id': '67de167f3eb34ddb88e44effaf746d27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 14:20:59.695716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80347) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:59.716767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:59.717781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application yield resource Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise self.value Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application session.commit() Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:20:59.718735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:20:59.719622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:59.720616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:59.722912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [parameters: {'id': '67de167f3eb34ddb88e44effaf746d27', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:59.724848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 14:20:59.726887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:20:59.730797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 226/478] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Jul 12 14:20:59 2022] POST /identity/v3/roles => generated 144 bytes in 269 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:59.731678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.825528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.859198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14d3e9-c10a-40d5-8da9-22cf4e644ef3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.897120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.898442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.899002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.899473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.911311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-6fa06be2-d9e3-4858-9f21-aebb342d7b4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:59.921463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:59.922382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:20:59.923182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:20:59.925637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 253/479] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:59.932034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:00.041372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19ddb6-249f-4a8d-8cff-c8cd216d572f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:00.072127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:00.081084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:00.082094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:00.088708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:00.088708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:00.088708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:00.088708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:00.088708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe420a-84c7-41b4-833f-d5966a091b08 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:00.094902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 227/480] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:01.111676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3353e1-c2d7-48ae-b7de-b1add4fa57a0 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.112198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3353e1-c2d7-48ae-b7de-b1add4fa57a0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.112274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3353e1-c2d7-48ae-b7de-b1add4fa57a0 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.113906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 254/481] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:01.119403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0d6e-93c9-4e24-9294-cfc3e03fb05e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.119823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0d6e-93c9-4e24-9294-cfc3e03fb05e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.119967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0d6e-93c9-4e24-9294-cfc3e03fb05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.138342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a51f0d6e-93c9-4e24-9294-cfc3e03fb05e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:01.139538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a51f0d6e-93c9-4e24-9294-cfc3e03fb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:01.225915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 228/482] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:01.274651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3353e1-c2d7-48ae-b7de-b1add4fa57a0 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:01.437243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3353e1-c2d7-48ae-b7de-b1add4fa57a0 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:01.563540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:01.571587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903db2c-14db-4fc4-8e0c-efa156249744 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.571902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903db2c-14db-4fc4-8e0c-efa156249744 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.572188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903db2c-14db-4fc4-8e0c-efa156249744 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.573718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 229/483] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:01.578870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:01.579669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.579867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c396aa-e7a6-4249-84c0-978674a0dfab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.579970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.580174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c396aa-e7a6-4249-84c0-978674a0dfab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.580272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.580450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c396aa-e7a6-4249-84c0-978674a0dfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.601023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-42c396aa-e7a6-4249-84c0-978674a0dfab None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:01.602410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-42c396aa-e7a6-4249-84c0-978674a0dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:01.603643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:01.611378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:01.612482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:01.613335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:01.617593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 255/484] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:01.623655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:01.680131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 230/485] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:21:01.690278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04e6f-dc35-4ddd-af7b-a92d4243c322 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.690808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04e6f-dc35-4ddd-af7b-a92d4243c322 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.691309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04e6f-dc35-4ddd-af7b-a92d4243c322 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.696569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 231/486] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:01.697907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.698395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.698868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.716888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:01.718138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:01.764737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:01.785188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbf5e-ffa9-4268-bb46-b3a4022e16a9 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:01.786145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 232/487] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:01.787110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:01.826799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:01.827477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.827763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.828088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.829330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:01.829654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:01.831096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:01.843017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 256/488] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:01.843865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:01.911532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:01.917339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f1d9d-b2ad-4901-acbd-7bff2ebb5b71 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:01.961854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:01.964143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:01.964613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:01.965300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:01.977183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf07a73-76b9-4096-9202-db80a77fca1e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:01.981886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:01.991952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:01.992953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:01.993844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:01.997224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 233/489] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:01.998249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.024167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.025010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.025878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.026157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.039285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:02.048719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:02.049693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:02.050544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 257/490] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:02.051211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.055056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.121766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba1dc2-d660-433a-a240-3a45aa6b3014 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.157795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.158680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.158966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.159241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.163850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6747-14f6-47cd-88c5-3b23d5ad31f5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.170299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.171885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.174318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:02.189997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:02.191111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:02.192011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:02.192011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:02.192011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.192011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:02.192011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 234/491] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:02.192523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.206493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.207217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.207500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.207823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.208683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.208993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.210853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.221232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 258/492] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.222182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.296637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.305553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742af0f-aa89-4735-be73-6debc8c699fc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.331204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04a3aa-11c9-4005-b798-47f19b95be57 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.342384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.343070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.343715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.344196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.345389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.346191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.347583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.361339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 235/493] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.362326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.373128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.373807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.374092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.374367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.375090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.375398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.376749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.394177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 259/494] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.394177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.443898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.497599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87ceb-27ba-44a6-81fd-f89e0bff60cb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.517607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119ef09-a5ec-46c9-88c5-6b665834a8e4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.531736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.532532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.532817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.533100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.543817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.546049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.555766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.556050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.556855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:02.563792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:02.564634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 236/495] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:02.570573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.574236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.575784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.577283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-b146fa42-e82d-465e-b10c-15d56178712a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:02.594256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:02.595170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:02.596026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:02.596026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:02.596026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:02.596026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:02.596026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 260/496] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:02.597122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56e30c-b01a-4cd7-ab55-3702d70925bb None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.597506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56e30c-b01a-4cd7-ab55-3702d70925bb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.597849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56e30c-b01a-4cd7-ab55-3702d70925bb None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.599339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 261/497] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:02.603889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56e30c-b01a-4cd7-ab55-3702d70925bb None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.682384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2cc52-f033-4a74-a2cd-6777c6cf2192 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.692500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.709318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56e30c-b01a-4cd7-ab55-3702d70925bb None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.717693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.718494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.718780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.719058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.726947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.728415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:02.744664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:02.745664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 237/498] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:02.751366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.752025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.752572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.752926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.753330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.754119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.754539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.756002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22685a6-808e-4d8b-880b-f518be8db2c9 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.772931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 262/499] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.772931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.772931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.772931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.787601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:02.788786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:02.821726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.855801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 263/500] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:02.856352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1d1ae-b58e-4b98-b986-ca0d271fc637 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.856634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.899112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.899771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.900076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.900355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.901071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.901406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.902953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.914478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 238/501] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.917819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.930292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.955204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3a453-c4f5-4251-a1cd-f356e70447bf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.988192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.988861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.989142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.989448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.990281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:02.990590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.992026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826267-49f9-4762-9076-45a29dc325fa None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.016730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfde01e-2af5-4d47-96e6-7b487f3c750a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.028053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 264/502] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:03.029428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941cae-d009-41f7-99e1-ec0f1993a736 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.029719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941cae-d009-41f7-99e1-ec0f1993a736 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.030002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941cae-d009-41f7-99e1-ec0f1993a736 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.031232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 265/503] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:03.033305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a941cae-d009-41f7-99e1-ec0f1993a736 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.046410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.054942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.069568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.069568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.069568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.069568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.069568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:03.079222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:03.080103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 239/504] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:03.081283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.134521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a941cae-d009-41f7-99e1-ec0f1993a736 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.164023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.170049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.170860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.171146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.171427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.181315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:03.185138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0236a-6ef4-43fe-8871-095414ff15bf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:03.188947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:03.190089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:03.190920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 266/505] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:03.191637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.219533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.220317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.220603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.220878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.227946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.229986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-2681e986-4d3a-4377-8ae3-1170dfddb8bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:03.244101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:03.245045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:03.245994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:03.249445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 240/506] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:03.250577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.250872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.251149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.267101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:03.268227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:03.293797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3bf47-e414-4539-9a9b-29dc322d8ada None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.309286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.326033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.326673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.326958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.327233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.332704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 241/507] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:03.333534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.390329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.399473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.401482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.434813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03674ab9-6b56-47c7-9c8a-9a0cdd77ec46 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.470472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.471142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.471423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.471698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.472534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:03.472841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.474314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.503479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 267/508] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:03.504385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.510701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 242/509] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:03.511523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.545345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.595000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.603440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b83060-c7c9-461a-9b9f-46534dd708d5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.608446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a1c49-8b14-440e-8a86-2a9927eb4faa None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.637026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.637967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.638254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.638534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.639276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:03.639583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.640948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.641999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.642669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.642958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.643247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.644078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:03.644392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.645862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.649185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 268/510] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:03.650532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.667979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 243/511] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:03.667979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.749831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619892a-5ef6-479c-b5f8-7275fd32c328 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.754604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.763636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdcf0f-9dcf-4d1c-9a83-288a9f20451a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.783857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.784638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.784920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.785220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.795730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.797105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.797231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.797979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.798258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.798532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.808308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.809762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:03.813975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:03.814943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:03.815853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:03.815853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:03.815853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:03.815853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:03.815853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 269/512] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:03.816116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.826548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:03.831500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:03.832552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 244/513] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:03.833657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:03.922199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f953b07-2059-42bb-b6bd-85ae9cc783f7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.932686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f0d9e-fae1-4e58-a77b-bb6738e873e7 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:03.954532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.955142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.955311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.955586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.969910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:03.970703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:03.970997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:03.971304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:03.982243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:03.983689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:03.986477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:04.002221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:04.003098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:04.003956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:04.003956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:04.003956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.003956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:04.003956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 245/514] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:04.004804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.028413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.029926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.037441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.102318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8255b-9311-416b-9002-0d5800e5fa15 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.126332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 270/515] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:04.127151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.136370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.137020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.137595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.137875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.199069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.205379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.206838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.225413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1f5e9-2eb2-45f2-b6d1-5845e50f9e44 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.234367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.261583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.262495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.262786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.263566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.264576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:04.264885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.266308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.277513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 271/516] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:04.278300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.313675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 246/517] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:04.314801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.383669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d0c0-e857-4db8-a49a-995e20fc8377 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.408881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.414452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a1474-8c37-4008-9961-1abfdd16b33a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.416160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.416818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.417108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.417443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.418157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:04.418463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.419811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.432680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 272/518] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:04.433532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.450410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.451160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.451464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.451932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.452824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:04.453485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.456685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.462886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.466193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 247/519] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:04.466971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.534447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a5396-cbbc-4cd2-beb2-6fa6a11820c5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.567453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abfdc2-e0e5-4b9a-a3f2-d8ac4c5ce719 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.569398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.570192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.570476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.570754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.580897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.582375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.601191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.601883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.602163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.602437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.603689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:04.603787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:04.604781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:04.605676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:04.605676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:04.605676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.605676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:04.605676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 273/520] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:04.605965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.606032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.676216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 248/521] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:21:04 2022] POST /identity/v3/users => generated 247 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:04.677521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.685656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.696713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.710410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3897b1-d3c7-4e54-a539-a982d91a51f3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.749232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.750359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.750647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.750922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.763074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.764553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:04.784834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:04.786013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:04.787083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:04.787083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:04.787083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:04.787083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:04.787083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 274/522] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:04.787371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.797665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73040c70-f98b-40b9-9d6d-b29902ae85fb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.838172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.839093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.839301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.839534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.846553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.848295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:04.863921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:04.864823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:04.865750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 249/523] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:04.866653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:04.906509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbe89-6e34-4733-9fbd-6b9cd9962a30 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.936725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.942725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:04.943629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:04.948386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:04.948386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:04.948697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:04.957337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:04.962451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:04.977654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:04.978808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:04.979939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:04.980545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168bab5-803e-4d1f-9be3-26a57b705a02 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:04.980619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 275/524] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:04.981456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.020200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.020860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.021141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.021442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.022162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.022573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.023938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.041333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 250/525] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.042273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.088776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a25ba-6f42-4735-a284-ae081b3fe5ec None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.124208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.125366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.125733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.126159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.126894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.127275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.128721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.146599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 276/526] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.146599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.146868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf6d0-b77c-429c-bc22-d14d3ebba5a4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.178718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.184994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.185682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.185968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.186245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.187086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.187343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.188778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.200146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.207091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 251/527] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.207745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.263521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77d2a-b546-4272-8fec-acb915691bcc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.300287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.300950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.301264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.301546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.302384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.302927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.304114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.311351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f9df-08a5-40e7-b179-b931332abc7f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.323195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 277/528] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.323879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.354879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.355666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.355944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.356221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.362914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.364362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:05.377801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:05.378666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 252/529] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:05.380125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.437549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b6a86-c394-4558-89b1-fe1ca06e6f5d None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.447255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.449460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.475549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.476338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.476625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.476902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.483708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.485167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.500087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92b0a-8636-40f5-a69e-6f57466acb00 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:05.500958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:05.501869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:05.502743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 278/530] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:05.503210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.539972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.540645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.540928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.541226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.542053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=401f40d749db4643824d38db0cf533cf) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.542366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.543781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.549819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 253/531] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/role_assignments?user.id=401f40d749db4643824d38db0cf533cf&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.550688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.655940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a7310-da9f-4575-b9f4-84543934625f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.662349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9c5b-894e-46aa-955d-4123dc562406 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.682642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.701862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.702576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.702928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.703281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.704174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.704577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.705422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.706256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.706448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.706894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.707337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.724889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 254/532] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.725977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.761581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.797077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.799470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.835296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ced74-19e8-4920-be2b-fe59d389dc01 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.875670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.876322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.876838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.877086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.878020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:05.878399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.879912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.897551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 255/533] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.898551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.932791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 279/534] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:05.935086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.944154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:06.022837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:06.029929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b552e5-e23f-4e2d-b666-eb1698679df7 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:06.070081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8a9c-bed8-4356-85e2-ea0c7482544e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:06.079838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:06.080696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.081420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.081802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.083053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:06.083713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:06.085446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:06.092740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 256/535] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:06.093684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:06.124508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:06.125366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.125830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.126205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.127162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=177ef85784a94f3782aa48b60a6fb8cc) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:06.127585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:06.129206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:06.136219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 280/536] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/role_assignments?user.id=177ef85784a94f3782aa48b60a6fb8cc&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:06.262682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:06.265709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74688518-cadb-4074-9232-b41dd6630e61 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:06.323005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:06.323736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.324016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.324290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.342418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:06.343876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:06.366719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:06.367633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:06.368543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:06.368543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:06.368543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:06.368543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:06.372205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 257/537] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:06.373470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:06.551664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d743b24-43d5-4d68-ae4a-7ab30b003f39 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:06.598262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:06.603702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:06.604366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.604647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.612490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.612490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:06.612490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:06.612490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:06.624607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 281/538] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:06.629636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:06.767458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e5b8b-0d48-4367-8aa7-33736b7bd12f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:06.806580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:06.807733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.808239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.808710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.819872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:06.821694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ba76b84f-a741-4faa-847b-aece66f2a0ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:06.835963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:06.836915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:06.837847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:06.840486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 258/539] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:06.850334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:06.967134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83c96-b9ab-46e7-8b00-e7af81db4a9f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.967453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83c96-b9ab-46e7-8b00-e7af81db4a9f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.967689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83c96-b9ab-46e7-8b00-e7af81db4a9f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.969318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 259/540] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:06.975063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:06.975352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:06.975632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:06.997816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:06.999009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:07.010418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035682f-4d78-4b9f-8db6-24684343b96b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.014074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.045911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.046578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.046859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.047237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.048028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:07.048346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.049927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.068200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 282/541] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:07.073990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.146839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 260/542] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:21:07.153904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.172187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.242647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9756-7607-4650-ae99-7d5eeb944a22 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.309779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.310835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.311388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.311877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.312942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=78ea4fd4eacb4664af0dc24ebfcacd42) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:07.315016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.316669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.328513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 283/543] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/role_assignments?user.id=78ea4fd4eacb4664af0dc24ebfcacd42&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:07.365351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290f929-2a20-4c29-9224-532676f9a5a0 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.423190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.424230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.424758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.425204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.447418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:07.454472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:07.455557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:07.456568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:07.459061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 261/544] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:07.465977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.580462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91218-0779-474e-a331-1eec4345f1d8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.598975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.618113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.619144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.619575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.619993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.620882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:07.621380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.623020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.633848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 284/545] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:07.640522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.645474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.749326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0940-e29d-4a6f-890f-816424db5e7c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.783487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.784677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.785184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.785670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.797376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.799095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f042ccaa-3f1f-4dc5-b207-e5740dc2b7f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:07.816326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:07.817223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:07.818196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:07.818196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:07.818196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:07.818196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:07.820386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 262/546] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:07.826151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.932515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7edb44-fc68-4e79-a8e8-1831e923c978 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.940230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec0e2-afcb-43c2-92ff-06e60dfe81a0 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.940776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec0e2-afcb-43c2-92ff-06e60dfe81a0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.941304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec0e2-afcb-43c2-92ff-06e60dfe81a0 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.942863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 263/547] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:07.947970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.948260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.948543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.974991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.976002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.976514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.977454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.979084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:07.979650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.981308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.986242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:07.987460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:07.994994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 285/548] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:08.000585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.043814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.107252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2c28e-b10b-4fa7-9070-0d47611e617f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.133524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.145076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.145908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.146194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.146471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.160024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.162946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.170381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 264/549] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:08.184497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:08.185497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:08.186441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:08.189030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 286/550] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:08.194808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.208655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.338280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.342618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee596574-9956-420c-aa4e-ea6e184c4430 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.377885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.379475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.379970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.380446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.392503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:08.401259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:08.402360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:08.403194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:08.405909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 287/551] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:08.410821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eacfc-31da-44ad-a34a-a1810d499274 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.412175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.482922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.483649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.483985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.484256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.485112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:08.485660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.488062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.522185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 265/552] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:08.528845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.579854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ad11-b775-412a-9c2d-1c279372d833 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.603664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.615780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.616448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.617108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.617108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.617754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:08.618060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.619956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef9485-62e1-44e0-b429-0ce28929266a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.628080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 288/553] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:08.629440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee757c-6301-4bcf-a857-32de397c8d53 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.629727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee757c-6301-4bcf-a857-32de397c8d53 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.630005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee757c-6301-4bcf-a857-32de397c8d53 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.633208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 289/554] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:08.633208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493a326-853d-4537-a567-9a71010da3cb None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.633208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493a326-853d-4537-a567-9a71010da3cb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.633208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493a326-853d-4537-a567-9a71010da3cb None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.634879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 290/555] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:08.635591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493a326-853d-4537-a567-9a71010da3cb None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.661773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.672865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab40-d9be-449b-80b5-e0d7e82b5c6b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.707889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.708534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.708816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.709089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.742292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493a326-853d-4537-a567-9a71010da3cb None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.781088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.781977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.782547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef3289-f5ce-4111-8ce4-acfda56e00f8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.782770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.783052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.783326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.795259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.796714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-12e63f5c-c443-48d2-9af7-07f80e8fe419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:08.815878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:08.816780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 291/556] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.818107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.832397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.835309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:08.836496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:08.897701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 266/557] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:08.899151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c295a84-7b45-4149-a987-a622420735fe None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.899443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c295a84-7b45-4149-a987-a622420735fe None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.899752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c295a84-7b45-4149-a987-a622420735fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.921176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 292/558] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:08.921176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.922667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9c295a84-7b45-4149-a987-a622420735fe None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:08.923602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.923983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9c295a84-7b45-4149-a987-a622420735fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:08.998121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 267/559] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:09.006301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c295a84-7b45-4149-a987-a622420735fe None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.023115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd951dd-7b2e-461f-b2cf-9fd907fae5bf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.059202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.059972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.060345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.060735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.061488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:09.061796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.063226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.080259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 293/560] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:09.080259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.082671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.112131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c295a84-7b45-4149-a987-a622420735fe None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.148253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.149007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.149319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.150118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.152378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.160497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.162180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:09.180815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.181797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.182753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:09.182753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:09.182753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:09.182753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:09.185314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 268/561] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.186104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.193112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2ca7-42f7-4e4f-90a0-17ca0006f4f4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.229618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.230414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.230696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.230976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.241641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:09.249775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.250829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:09.251698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 294/562] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.253085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.299535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552c4be-8189-4107-97f5-c47d19aab1a4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.323678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.334390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.335213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.335553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.335830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.346622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:09.354888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.355837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:09.356782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 269/563] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.357841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.363652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5901dd-a84c-40b8-b7f1-98fb4f609b2e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.387374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.400329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.401135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.401514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.401790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.410254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.412028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:09.427636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.433743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 295/564] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.434711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.520321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c8b07-7301-454d-b8ba-a7a84564ead0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.554213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.554888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.555174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.555456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.556183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:09.556494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.557881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.575117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 270/565] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:09.575117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.594914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e352a-0fb5-498d-b2be-24a737382a85 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.620936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.629635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.630507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.630790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.631061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.631808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:09.632111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.633577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.642770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 296/566] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:09.647810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.678615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a72b65-8dd4-442e-b72b-4916251b69e9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.697678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.716007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.716677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.716959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.717261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.718006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:09.718309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.719712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.728238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 271/567] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:09.729041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.758234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae491d-855a-443f-820f-92c3c5bfffc2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.796934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.797637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.797920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.798196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.799026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:09.799336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.800761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.818729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 297/568] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:09.819441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.832727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bc9c7-bffa-42c1-b580-5ab783ce14b6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.840570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.866409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.867198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.867480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.867829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.874709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.876216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:09.890856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.891764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:09.892688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 272/569] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.893557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.918247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc41bdf-9263-421a-bcbc-78545f58e025 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:09.919834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:09.953184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:09.953978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.954259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.954534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.964500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:09.965971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-4b6b8615-7028-4cda-825d-4fc165769962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:09.982977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:09.983956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:09.985070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:09.985070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:09.985070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:09.985070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:09.985070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 298/570] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:09.985694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3770a2-8d0d-44a1-a0a8-37dc49352765 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:09.985985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3770a2-8d0d-44a1-a0a8-37dc49352765 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:09.986264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3770a2-8d0d-44a1-a0a8-37dc49352765 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:09.987678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 299/571] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:09.988186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3770a2-8d0d-44a1-a0a8-37dc49352765 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:09.997062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97df2b-fd2e-436d-b50d-5807a0e9224e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.031511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.032301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.032583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.032861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.042764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.044204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:10.061746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.062910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 273/572] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.063997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.082312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.090586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3770a2-8d0d-44a1-a0a8-37dc49352765 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.123951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.124741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.125024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.125330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.135788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.137529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:10.155187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.156153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.157063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:10.157063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:10.157063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:10.157063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:10.157063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 300/573] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.157491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.165409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.165940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbc3c5-11ea-4a37-9bee-7c9b7703aef9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.200027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.200683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.200968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.201263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.202053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.202362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.203837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829fad-4258-4403-8917-9eff22942326 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.214958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 274/574] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.216128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.216419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.216697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.232776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:10.233992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:10.256091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5396-69f5-4ee2-9de0-29510ce1a23a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.289965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.291158 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.291442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.291721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.292435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.292744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.294141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.300721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 275/575] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:10.301559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.310105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 301/576] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.310105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.312716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.375408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.401121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5a066-47b8-47ad-b831-90eff7210c34 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.410183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134265-beb5-4d60-9f53-b10385be85b5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.439888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.440561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.440848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.441127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.442054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.442179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.443556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.446843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.447502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.447784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.448058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.448768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.449072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.450566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.455155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 276/577] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.455895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.462854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 302/578] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.463720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.535816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.567853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8ca63-8c9c-42a3-984c-519908efebb6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.569296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072402c5-2b46-45b9-a44c-d92aa782b516 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.592814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.601594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.602153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.602328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.602425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.603225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.603330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.603500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.603590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.604347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=59997304919b432184f842b10ab9bba8) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.604664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.606150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.610531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.611559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 277/579] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/role_assignments?user.id=59997304919b432184f842b10ab9bba8&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.611946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.612327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:10.628230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.629497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:10.630663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 303/580] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.631356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.741346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9f0bc-a08f-4ad7-bfdd-addb6561ba9a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.757350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46010-93fb-45b0-bae3-3e9ab9afa93d None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.773051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.778410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.779204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.779483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.779754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.790096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:10.796324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.797126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.797474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.797754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:10.798177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.799099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:10.800049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 278/581] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.801446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.808050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.809523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:10.824775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.825850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 304/582] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.826824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.878755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.911371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327c412-512b-49ed-bde0-8844387279f3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.937439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d975ec-454b-4fae-be44-876907fc7190 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.945847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.946682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.946963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.947237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.953976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.955402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:10.969905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.970820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:10.971760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 279/583] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.972891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.979392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.980083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.980487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.980771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.981655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:10.981971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:10.983513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.003539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 305/584] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.008415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.029523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.080040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27974090-e4cb-495c-8388-7c7b94e64d95 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.116155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c08a4c-47c6-4a9b-a4e8-9f2eebb583b5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.119482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.119642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.120701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.121065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.121294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.122052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.122363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.123735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.133062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 280/585] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.134550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.157637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.159135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.159577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.159857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.161008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.161362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.162777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.186488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 306/586] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.186488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.239700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe498a8-315c-4b43-a1db-da7f2e2ef17f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.253983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.274008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.275349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.275620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.275896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.276686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.277003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.278473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.295693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 281/587] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.295693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.299433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c49fe-d842-43d4-9b28-03011051a4c2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.337571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.338375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.338652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.339357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.350896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.352339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:11.369781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:11.370689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 307/588] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:11.376403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.391695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.404222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582403ee-3c18-46a3-bd84-2c76f16b7e25 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.448623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.449448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.449730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.450004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.460928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.462497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:11.480072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:11.480966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:11.481995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:11.481995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:11.481995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.481995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:11.484611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316594a8-a0cd-481e-9214-cad2a2ce7065 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.487085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 282/589] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:11.488414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.523346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.527561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.528618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.528905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.529203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.540637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.542113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:11.561723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:11.562698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:11.563600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:11.563600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:11.563600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:11.563600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:11.563600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 308/590] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:11.563898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.605325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473c816-2992-4e56-bbdd-bcd0bb29d654 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.632929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.640533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.641464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.641751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.642028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/401f40d749db4643824d38db0cf533cf/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.656385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=401f40d749db4643824d38db0cf533cf, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.657766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.670431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8820d-1ed6-4b2f-ba84-97ec9e1d6969 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.705271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.705949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.706232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.706509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.707314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.707626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.708999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.720772 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 309/591] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.721598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.751165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 283/592] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:11 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/401f40d749db4643824d38db0cf533cf/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:11.752387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.848517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.849692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e2fa-3691-413f-b17f-5d5ccfedf1b7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.875790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309a21-e599-4136-b175-9d7c0b850454 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.879613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.892342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.893015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.893343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.893625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.897830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.897830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.897830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.910176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 310/593] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.911167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.922929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.923707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.924431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.924717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.925847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.930301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.930301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.943795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 284/594] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.944976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.034636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7cffd-018f-4d71-ba0f-c1078060c999 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.078067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8efd-afb9-4816-8b5c-6ae896a17565 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.079797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.081552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.081849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.082139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.090321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.092466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:12.111743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:12.112811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:12.114017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 311/595] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:12.114605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.121351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.122281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.122662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.123035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.127212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.134848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.137497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:12.155629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:12.156828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:12.158029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:12.161676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 285/596] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:12.161676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.177128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.239261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434932-3e52-4d5e-bf5e-6a1486e18401 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.281426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.282346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.282730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.283101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.287986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a9db-3521-40c5-99b6-7e2c428c604c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.292009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.292946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:12.308202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:12.309372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:12.310480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:12.316707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 312/597] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:12.317701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.332418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.333626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.334061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.334963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.336220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:12.336698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.339448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.356089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 286/598] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:12.357122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.418244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.435680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ee286-1921-42aa-84ab-23aafcb53146 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.443789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.463205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f6a4f-7812-4169-a9a3-3f7c84593a9a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.472134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.472883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.473182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.473481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.474326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:12.474636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.476071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.495493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 313/599] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:12.499445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.503838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.504508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.504794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.505070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.505902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:12.506208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.507701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.525996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 287/600] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:12.526903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.601644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fd948-7ae3-4e14-a1a3-2d7c6e6387c0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.633675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec8b8-95ac-4434-a47e-7d5648419f8a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.637984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.638875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.639157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.639435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.650933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.654419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=a6b4ed458d984a51829114a73534fce2, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.655940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e88be9-70fa-4544-afce-50f42b1c3a9a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.670429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.671166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.671452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.671726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.683210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.685411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:12.701204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:12.702093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:12.703076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:12.703076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:12.703076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:12.703076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:12.703076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 288/601] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:12.703807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.737823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 314/602] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:12 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:12.739434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b4ea-8ebc-40c1-960a-8c1a1971a60d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.739743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b4ea-8ebc-40c1-960a-8c1a1971a60d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.740002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b4ea-8ebc-40c1-960a-8c1a1971a60d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.741378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 315/603] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:12.742018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b4ea-8ebc-40c1-960a-8c1a1971a60d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:12.759607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:12.827936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3549de-1d32-4102-9297-9548bfb6bd89 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.865654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b4ea-8ebc-40c1-960a-8c1a1971a60d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:12.872407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.873625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.874017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.874398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/177ef85784a94f3782aa48b60a6fb8cc/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.892219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=177ef85784a94f3782aa48b60a6fb8cc, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.894013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.908430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:12.909114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.909433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.909713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.914337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:12.914337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:12.914337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f3388-744b-486a-b330-3fe5c45ede6d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:12.925918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 316/604] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:12.927287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:12.927571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:12.927848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:12.947458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:12.948643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:13.003163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 289/605] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:12 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/177ef85784a94f3782aa48b60a6fb8cc/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:13.008856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.048342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 317/606] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:13.051174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.091499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.095375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.224170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91c7-a6aa-461e-a5ad-abc90e034bf6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.236733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006502b-52a9-4734-8378-cf6be38e83a0 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.280262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.280485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.281287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.281581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.281858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.282206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.282870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.283338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.289345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.290789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.296822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:13.305074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:13.306104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:13.306951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:13.307593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:13.308396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:13.309327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:13.309753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 290/607] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:13.311256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 318/608] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:13.317058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.318653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.401667 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.445300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.445776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab5316-7dd2-47f1-81d4-e12c109d0f46 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.484835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.485595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.485889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.486169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.486929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:13.487238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.488790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.502946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 319/609] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:13.503526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32bc97-afe7-4b54-85e3-b12315578f74 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.504395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.558170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.558668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.558957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.559239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.560040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:13.560356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.561833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.575228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 291/610] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:13.580867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.615372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2adfb-9aae-4a7d-ae28-8a167134335f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.669255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.679722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.681184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.681478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.681756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.697658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691f9e9-4adf-455e-b5e2-32fa3dc37268 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.704372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.706045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.733531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:13.734262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:13.735166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:13.736081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:13.736081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:13.736081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:13.736081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:13.736304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.736304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.736304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/78ea4fd4eacb4664af0dc24ebfcacd42/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.736496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 320/611] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:13.740893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.748648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.755862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=78ea4fd4eacb4664af0dc24ebfcacd42, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.757445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915a9c9-aab6-4932-a788-c86c540f471b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.840974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 292/612] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:13 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/78ea4fd4eacb4664af0dc24ebfcacd42/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:21:13.890127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829af053-bae4-468f-961e-3cb0ce17294f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.942808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.943566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.943855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.944135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.944930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:13.945265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.947278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.961006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dd162-fc5f-4adf-94be-808a5607d0d2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.961363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dd162-fc5f-4adf-94be-808a5607d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.961617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dd162-fc5f-4adf-94be-808a5607d0d2 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.963336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 293/613] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:13.967645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 321/614] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:13.968841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.969100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.969414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.974715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.997899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:13.999439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:14.030239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.093374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 294/615] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:14.109716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faef39-f711-40a1-aca9-5eded1ca0cfe None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.117387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.149507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.150311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.150593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.150867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] PATH_INFO: `/v3/users/swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.159059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.160793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:14.177753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:14.178735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:14.179692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:14.180234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 322/616] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:14.186940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.292336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.296108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a27ab-1a96-4f2b-9071-e4b5188dfede None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.332544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.333361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.336495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.336495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.336495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:14.336495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.336852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.339128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231df59d-fac0-4efa-8a4a-4098bca1b021 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.359919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 323/617] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:14.361310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.374914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.405267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.406078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.406359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.406672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.435859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:14.449147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:14.450235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:14.451091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:14.454180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 295/618] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:14.462657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.479878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a3b69-427e-4349-bffc-5349f13da742 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.519987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.521539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.521826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.522094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.542842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.546145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:14.577289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:14.578267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:14.579180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:14.579180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:14.579180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:14.579180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:14.579180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 324/619] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:14.585596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.606800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.634161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017203-f2b6-468c-be94-4466c788b6b3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.698961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.699739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.699963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.700248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.702321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:14.702641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.705447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.724667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 296/620] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:14.730678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.766871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50a5f-4cc6-4b41-90f3-b967e9709409 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.803788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.830688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.831362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.831647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.831923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.832662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:14.832952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.834367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.839469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf831a-79e4-4c81-b4ba-b152c0ae9d4c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.856939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 325/621] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:14.863923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.881026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.882274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.882563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.882846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.896021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.897566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:14.917413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:14.918327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:14.919261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:14.919261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:14.919261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:14.919261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:14.919261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 297/622] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:14.924252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.014904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.076393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.085563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642be1-f4da-4802-baf3-d5d89f7315b5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.088515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12265a-fd4a-4430-88a4-49118b4e73ae None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.122899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.123567 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.123851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.124133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.124970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:15.125313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.126677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.140581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 298/623] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:15.145931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.154418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.155236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.155522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.155801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.168588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.170084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:15.200880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:15.201925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:15.202838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 326/624] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:15.214219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.260946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1822f-abe1-439a-9fa3-272c4fc3004c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.298816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.299621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.299904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.300181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.308757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.310248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:15.326989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:15.329431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:15.332314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 299/625] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:15.333972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.349093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.414488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47618-bafc-41ab-8c84-ef2f7e8acf14 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.443170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.454775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.455588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.456330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.456682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.457876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:15.458260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.459695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35df1b2-91e6-4e3f-9200-b89d9f784640 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.459809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a683b0-7ddc-4063-9647-d25bbc3e19d4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.477243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 327/626] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:15.478856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9e59-b6a1-4c98-800c-b9c9ed4ea3f1 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.479338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9e59-b6a1-4c98-800c-b9c9ed4ea3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.479860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9e59-b6a1-4c98-800c-b9c9ed4ea3f1 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.485319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 328/627] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:15.486313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9e59-b6a1-4c98-800c-b9c9ed4ea3f1 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.505317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.506153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.506427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.506791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.507710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:15.508171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.510492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3179-5b77-4363-b741-9b13e5b6a478 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.529859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 300/628] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:15.535745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.536118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.536466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.557550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:15.558947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:15.598363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9e59-b6a1-4c98-800c-b9c9ed4ea3f1 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.628613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.641178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 301/629] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:15.641178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.641964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ac249-abc9-4154-b926-585b3523840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.643096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ac249-abc9-4154-b926-585b3523840f None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.643577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ac249-abc9-4154-b926-585b3523840f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.644059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ac249-abc9-4154-b926-585b3523840f None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/59997304919b432184f842b10ab9bba8/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.673648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ac249-abc9-4154-b926-585b3523840f None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=59997304919b432184f842b10ab9bba8, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.675093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ac249-abc9-4154-b926-585b3523840f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.736929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7aa76b-dfd8-4fd9-a5db-7b7eb2d629df None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 329/630] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:15 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/59997304919b432184f842b10ab9bba8/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce930-6b0e-46ec-9068-be152a8a1b8b None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce930-6b0e-46ec-9068-be152a8a1b8b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce930-6b0e-46ec-9068-be152a8a1b8b None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 330/631] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:15.745353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ce930-6b0e-46ec-9068-be152a8a1b8b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:15.784771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.785470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.785751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.786070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.800606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.866549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:15.869556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:15.871086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:15.894422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ce930-6b0e-46ec-9068-be152a8a1b8b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:15.940697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:15.942016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.942016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.942213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.954376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2c40b74d-c8c8-4168-b32a-9943e4e5b734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:15.962864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:15.963844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:15.964722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:15.966766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 331/632] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:15.968319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:15.968888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:15.969415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:15.987260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:15.988798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:15.996512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 302/633] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:15.996512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.064899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 332/634] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:16.066178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.116666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.129065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.132390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9871d3-2038-4672-8f6a-fd50426e006f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.169715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.170853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.171386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.171864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.172842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:16.173380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.175153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.184030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 303/635] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:16.185241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.205484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12c93a-aaef-4b3c-8ddc-1cb718056721 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.241606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.242397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.242674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.242969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.253863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.255330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:16.273781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:16.275313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:16.276234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:16.276234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:16.276234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.276234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:16.276234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 333/636] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:16.276977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.298174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5eed5-1807-4dcb-b9a3-2a7f5e67fdfd None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.333261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.334060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.334345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.334623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.346046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-fa6aee27-a039-4cb2-929f-07c00586b141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:16.353776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.354991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 304/637] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:16.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa668297-5e6a-4df7-bbec-4033b7f74546 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.357444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa668297-5e6a-4df7-bbec-4033b7f74546 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.357444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa668297-5e6a-4df7-bbec-4033b7f74546 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.361851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 305/638] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:16.362601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa668297-5e6a-4df7-bbec-4033b7f74546 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.372530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.389041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.395618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc96681-2cb2-41ca-8116-391b0ac0f432 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.430833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.431635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.431921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.432195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.443149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.444625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:16.463766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:16.464765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 334/639] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:16.466242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.482819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa668297-5e6a-4df7-bbec-4033b7f74546 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.520212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.520897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.521191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.521678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.522207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:16.522516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.523888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca6ead-210a-4824-acc2-068531a1ce12 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.536789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 306/640] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:16.537877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.538168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.538450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.554778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:16.555998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:16.580737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568c90c-386d-41fc-bd83-c7cea0f3ae4e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.611794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.617499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.618707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.619001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.621589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.621589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:16.621589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.621835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.628984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 307/641] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:16.629623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.639886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 335/642] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:16.640512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.647726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.742045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bc394-4b03-431c-8aae-7ea165e4188f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.752684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7653b-f911-41c5-8482-6ad1c687fcc4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.779581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.780246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.780533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.780813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.781559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:16.781868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.783233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.788414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.789235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.790591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.790909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.798232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.799357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 308/643] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:16.799663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.799961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:16.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:16.817864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 336/644] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:16.818832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.879604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.880945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.912295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f789cbe-6b6d-4ad5-aef9-4b127f425830 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.926934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dab27-9f84-4da3-b43c-7ffd4ad7cf7c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.949174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.950400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.950688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.951107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.963511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:16.964414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.964963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:16.965303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.965590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.965867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.978244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:16.984952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:16.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:16.997589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:16.997589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:16.997589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:16.997589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:16.997908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:16.999072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 337/645] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.000220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.004073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 309/646] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.004073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.103042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db40b2-bbf2-40dd-877e-9f970e743eec None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.118766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb6af-7e42-470c-9e9a-fea079f6f064 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.126821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.143500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.143661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.143661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.143661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.143661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.144054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.144054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.144054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.155000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 310/647] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.155978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.164057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.165238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.165528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.165808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.174341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.175784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:17.189701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:17.194704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 338/648] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.195678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.263912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2740-3836-4a8e-9207-9fc6d62998cf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.299122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ebfdf-e9a0-4cba-8756-714dc8369b00 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.300413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.301076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.301391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.301675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.302458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.302769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.304188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.321526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 311/649] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.321526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.343923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.344723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.345078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.345454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.346274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.346657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.348091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.356434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 339/650] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.361420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.368435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.399934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.426276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e36e6-608d-4236-a77c-521ca2a37abb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.465530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.466485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.466918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.467201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.468035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=401f40d749db4643824d38db0cf533cf) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.468349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.469798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.476341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 312/651] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/role_assignments?user.id=401f40d749db4643824d38db0cf533cf&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.477604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631469fd-9d52-43f4-90ac-73d9ed3c2e22 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.478292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.524516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.525664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.526489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.528022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.529228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.529778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.531489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.559134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 340/652] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.560363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.617325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8acc4-e273-4c85-9772-bdb6229e9c4d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.629566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.667582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.668701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.669193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.669650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.679332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.681458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.692577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:17.696654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:17.697831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:17.698992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:17.700085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 313/653] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.700916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.707543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127d8b1-2382-44ca-8026-d3c2afe85d92 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.749471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.751141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.751428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.751709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.765517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.767435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:17.791998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:17.793149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:17.794277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:17.794277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:17.794277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:17.794277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:17.794277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 341/654] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.794634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.825066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7ade-b90c-453c-8613-2424c5123682 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.861198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.861865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.862146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.862425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.898675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.917919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31383c3-ac6f-488a-b1f4-1855ef07e13a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.933336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.934785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35934ab8-6d8c-4b69-a7c0-2126e4a4124d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.953651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.954325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.954614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.954893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.955736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.956055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.957508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.979795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 342/655] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:17.980669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.984120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.037061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 314/656] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:18.038278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34f3af-f97d-41ad-9ee5-4877f3a8784b None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.038572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34f3af-f97d-41ad-9ee5-4877f3a8784b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.038854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34f3af-f97d-41ad-9ee5-4877f3a8784b None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.040812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 315/657] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:18.045253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34f3af-f97d-41ad-9ee5-4877f3a8784b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.096021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097a2da-fa69-4e13-bd6e-493581ed4afd None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.119797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.132357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.133024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.133337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.133683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.134410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:18.134719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.136088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.148708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 343/658] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:18.149953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.160865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34f3af-f97d-41ad-9ee5-4877f3a8784b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.198611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.199268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.199555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.199837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.213893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.262896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943171ee-5f3c-49b3-8c76-7cd128b363d2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.269961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.271420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.300235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.301192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.301581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.301958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.313054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.314608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3a7618c0-2182-4246-a866-6bd4951973b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:18.331752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:18.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:18.333986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:18.333986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:18.333986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:18.333986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:18.338037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 344/659] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:18.339198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.339493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.339771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.344082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.356869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:18.358112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:18.383505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 316/660] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:18.383505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.445574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 345/661] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:18.446362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.491961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb247c-10fb-4682-bc1c-2350e169b414 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.500092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.528136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.528932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.529473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.529602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.536854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.538328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:18.552641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:18.553715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 317/662] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:18.554718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.560306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880e287-aa90-4f0a-87d8-b026e2c6d70b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.587403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.597661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.598324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.598609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.598886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.599602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:18.599966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.601376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.617509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 346/663] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:18.617509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.667948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920debb2-9d54-4360-a762-64638356fdbc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.702892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.703694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.703974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.704249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.715645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.717151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.723101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c6cd-d785-431f-a89d-ca6ae21d7d44 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.730934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:18.736033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:18.741586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 318/664] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:18.742812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.767128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.768631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.769360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.769787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.783304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:18.793531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:18.794669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:18.795731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:18.798215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 347/665] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:18.799616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.823934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:18.845857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af38c2-b9bf-4785-a9cf-b74566b726cc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.881147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.881852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.882133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.882407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.883244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:18.884089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.885535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.902915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 319/666] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:18.902915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:18.913428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208d565-636e-498f-9ebb-a9492cffefc6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:18.948691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:18.949503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:18.949784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:18.950057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:18.956841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:18.958319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:18.972073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:18.973245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:18.974207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 348/667] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:18.974736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.002057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.013740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c85b04-d8cd-4f7b-9842-9d59a178992f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.048640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.049421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.049775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.050502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.051364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.051686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.053807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.061100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.066767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 320/668] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.067416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.081060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe141d-408a-46f4-8bea-c74acd9b08ef None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.115276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.116047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.116443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.116763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.117636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.118541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.120006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.132317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 349/669] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.132317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.174624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9858b9-82a0-4450-99e4-3a1b605d1331 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.209794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.210533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.210885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.211248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.220467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.233863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f77c91-118d-45fe-9f66-ccd43d6bf467 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.271410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.272072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.272421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.272760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.273640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.274042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.275468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.277773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.281869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.283417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.293229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 350/670] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.293229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.393447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 321/671] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:19.393447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.403532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd87b5-55fb-4676-9b27-ac8bb0eba522 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.447211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.448014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.448293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.448560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.455505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.457437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:19.470807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:19.471888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 351/672] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:19.478091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.487060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.508263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.513707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbde31-ce8e-4f7b-92f4-156949e1e2e1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.553884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.554590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.554877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.555153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.565967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.567288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.580584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046687-9fda-421d-a925-6db15a3dd205 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:19.589201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:19.590325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 322/673] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:19.591439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.630144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.630804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.631086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.631364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.632337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.632668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.634945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.642049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 352/674] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.643160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.714355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9828e-7bdd-4b31-abc8-a68c0204d31c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.723922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.756838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.757274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba88e4-6406-4968-96dd-9d36cce853f8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.757690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.757977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.758254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.769732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.771198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:19.790892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:19.791871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:19.792779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:19.792779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:19.792779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:19.792779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:19.792779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 323/675] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:19.793064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.796341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.803595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.810025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cfd18-bb83-4a86-92ef-cec43b3284a4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.822100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 353/676] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.823550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f133-d4f9-4f1d-8629-2a3827f1c348 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.823841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f133-d4f9-4f1d-8629-2a3827f1c348 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.824116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f133-d4f9-4f1d-8629-2a3827f1c348 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.825624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 354/677] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:19.826285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08f133-d4f9-4f1d-8629-2a3827f1c348 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.914312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36319559-1291-4165-9b58-bc5a634ae467 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.938340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08f133-d4f9-4f1d-8629-2a3827f1c348 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.952326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.952988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.953338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.953850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.954332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.954638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.956056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.967884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 324/678] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.968585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.973668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.978741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.979701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.980085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.980753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.981463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:19.987464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.987464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc69c-f73d-4b54-b418-21576125f6de None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.000818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 355/679] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.000818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.000818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.000818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.017740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:20.019163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:20.034309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.074774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5387377-7d4b-4f5b-8508-20c1da005567 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.091487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 356/680] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:20.092419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.118773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.119751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.120204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.120479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.121352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=177ef85784a94f3782aa48b60a6fb8cc) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:20.121664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.123090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.129194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 325/681] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:19 2022] GET /identity/v3/role_assignments?user.id=177ef85784a94f3782aa48b60a6fb8cc&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.132438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.234958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c928b7b-33f1-4367-b9e6-3ad89d667036 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.266496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.273316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0f721-ca0c-4857-9a1b-7dea68118a0a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.282014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.282908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.283818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.283818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.298118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.298833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.305888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.317929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.318305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.318305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:20.318489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:20.319370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:20.320556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:20.321360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.325374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 357/682] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:20.326176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.327358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.328945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:20.346245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:20.347449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:20.348419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:20.353269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 326/683] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:20.354176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.469326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa086-cf7d-4dd0-8781-c3934a97ccca None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.471768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c938-193e-4928-aa7e-4b269fd2bc33 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.506315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.507112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.507578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.507687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.518423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.519858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.521088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.522626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.523140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.523627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.525223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:20.525771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.527902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:20.538268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:20.539227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:20.540716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:20.540716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:20.540716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:20.540716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:20.544820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 358/684] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:20.545693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.552616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 327/685] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.553763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.580994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.617902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.651934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396631-4341-420c-836a-dfabc31ee91f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.689253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.689937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.690218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.690493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.691281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:20.691589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.691967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d5e5-74f2-4d37-9866-829edb13c25d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.693018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.709645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 359/686] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.710507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.735891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.737527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.738181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.738770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.739973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:20.740875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.743021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.759408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 328/687] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.760543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.833739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.841569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef69637-788c-4187-ae8c-3dc4afa1c901 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.870530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b924a4-85f7-45d8-9e4a-b4af611fb97c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:20.873800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:20.878275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.878942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.879555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.879664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.908708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:20.909733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:20.910265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:20.910762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:20.911809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=78ea4fd4eacb4664af0dc24ebfcacd42) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:20.912327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.914016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:20.921084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 329/688] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/role_assignments?user.id=78ea4fd4eacb4664af0dc24ebfcacd42&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:20.922648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:20.956104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:20.957847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.033451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd448e8-12f1-42ee-9f82-5ca07b548200 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.061499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.073200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 360/689] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:21.076220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.079281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.084421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.084421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.084421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.084421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:21.089473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.187775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 330/690] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:21:20 2022] POST /identity/v3/users => generated 254 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:21.216013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f188-d576-4d83-8fea-9f53af4c97fb None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.264324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.265433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.265866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.266352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.279797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:21.281586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:21.306129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:21.307149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:21.308097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:21.308097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:21.308097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:21.308097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:21.308097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 361/691] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:21.313390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.347918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.431635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b978fe-b3bd-4a48-a36f-75662f449dc2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.473351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.474443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.474929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.475405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.476336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:21.476844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:21.478487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.492260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 331/692] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:21.497955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.554267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.607194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8a2e-88e0-410e-aa9b-9862da6d53d7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.643932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.644732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.645016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.645334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.652443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:21.653917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:21.668503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:21.669534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:21.670843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 362/693] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:21.675359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.765327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.782883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a63d0-9eae-44ff-bb54-bffc143158a6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.819400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.820064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.820631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.821484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:21.821795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:21.823215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c11c55-4c70-443c-9cae-cfb019d6cf9d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:21.835425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 332/694] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:21.841055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.937532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa1e3-3e9e-4c60-8bd3-ed848e4b686c None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.938104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa1e3-3e9e-4c60-8bd3-ed848e4b686c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.938104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa1e3-3e9e-4c60-8bd3-ed848e4b686c None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.940319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 333/695] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:21.945430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.945751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.946038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.961983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7e50b-80c9-42d7-90e7-b39ae92e04a7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.969475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.979210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:21.980393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:22.019607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.020277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.020562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.020843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.021719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=59997304919b432184f842b10ab9bba8) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:22.022037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.023476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.034177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 363/696] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/role_assignments?user.id=59997304919b432184f842b10ab9bba8&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:22.106984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 334/697] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:22.140036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.246590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ffb-a608-4c2b-839a-6cea43859b12 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.286059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.287228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.287719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.288210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.301471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-b130101d-d632-414d-9a45-2a7bb425bed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:22.309523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:22.310456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:22.311314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:22.314096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 364/698] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:22.319415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.395447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:22.422180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487653f0-369d-4356-8aee-32223d03ba4d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.456729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012357-1cea-4614-8b06-012f0928776e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.457366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012357-1cea-4614-8b06-012f0928776e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.457898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012357-1cea-4614-8b06-012f0928776e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.459755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 365/699] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:22.466305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.466600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.466882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.474467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.475437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.475887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.476302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.477255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:22.477717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.479239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.486605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:22.487810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:22.488915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 335/700] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:22.494520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.519890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:22.578979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 366/701] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:21:22.601079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.647100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff37d5-22a0-4479-a6e4-c383aa33755c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.703159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.703954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.704234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.704506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.718646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:22.719284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e808cd3-b633-4f9c-89cc-22cf88cef1fe None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.737034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.738553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:22.761081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:22.762311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:22.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:22.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:22.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:22.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:22.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 336/702] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:22.768279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.806468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.807529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.807958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.808373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.809191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.810720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:22.843400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:22.844452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:22.845952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 367/703] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:22.850188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.889264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206181f8-8911-4acc-bc2a-f3802a27ff2c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.915166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:22.929703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.930752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.931043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.931321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.937179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:22.937179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.937179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.948975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 337/704] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:22.954848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.021915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:23.078118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518670-90cd-4acf-9c72-7263ca629982 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:23.078309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9514ef0-c378-4221-85ba-d3651217f4eb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:23.117642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:23.118448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.118732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.119011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.130682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:23.137969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:23.142121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:23.143362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.144298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.144760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.145731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:23.146218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:23.147833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:23.153910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 368/705] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:23.161805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:23.162784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:23.163787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:23.165625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.168980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 338/706] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:23.168980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.301519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:23.308454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb6a2c-86d2-4bef-9110-e2cf6d30f864 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:23.343972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:23.344880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.345353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.345791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.346641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:23.347096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:23.348666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:23.348899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323768-3584-49d7-beaa-d13b633fce58 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:23.355136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 369/707] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:23.369562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:23.438741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:23.439442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.439726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.440002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.440858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:23.441180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:23.442620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709145f-819f-49a5-a09c-70f25a2e5659 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:23.481250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 339/708] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:23.486728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.621551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba068aa-ae2b-4093-9650-4aad5d7e886c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:23.660671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:23.661371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.661669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.661972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.663159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066a4a-b2c1-4181-a180-a92bb2e481a4 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.663482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066a4a-b2c1-4181-a180-a92bb2e481a4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.663743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066a4a-b2c1-4181-a180-a92bb2e481a4 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.665418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 340/709] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:23.673111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711780-724b-471b-9507-f5be63a338af None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:23.673413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711780-724b-471b-9507-f5be63a338af None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:23.673691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711780-724b-471b-9507-f5be63a338af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:23.688261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:23.706900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a5711780-724b-471b-9507-f5be63a338af None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:23.708171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a5711780-724b-471b-9507-f5be63a338af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:23.764244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:23.766097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:23.829565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 341/710] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:23.863874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711780-724b-471b-9507-f5be63a338af None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.936149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 370/711] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:23.943102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:23.997454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.000350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.004603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711780-724b-471b-9507-f5be63a338af None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.041632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.042712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.043140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.043556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] PATH_INFO: `/v3/services/network` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.044344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.045945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.057327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0030b17-8f98-4a84-af1e-09d295decef0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-6934c103-d723-4d5d-9d07-b1ca8bb1a240 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:24.060359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:24.061536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:24.062399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 342/712] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:23 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:24.063780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a7d2-aa4f-4214-86c8-7da3f2b41673 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.063780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a7d2-aa4f-4214-86c8-7da3f2b41673 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.063939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a7d2-aa4f-4214-86c8-7da3f2b41673 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.069120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 343/713] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:24.069120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741a7d2-aa4f-4214-86c8-7da3f2b41673 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.113054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.114568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.115287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.115776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.128317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.129977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-1f527bac-88f3-4a49-b581-57628a01fece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:24.148558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:24.150506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:24.151541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:24.151541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:24.151541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:24.151541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:24.153917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 371/714] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:24.155351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.155855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.156332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.175413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:24.176997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:24.189773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741a7d2-aa4f-4214-86c8-7da3f2b41673 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.227525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.228186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.228467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.228740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.229472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] RBAC: Adding query filter params (name=network) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:24.229779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.231277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.236717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 344/715] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:24.237653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.264181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 372/716] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:24.265398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.276592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.306680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.380938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea4f84-ed10-47f2-8e46-b2e23082a4b1 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.382850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd8b9-776b-420b-8214-09437796dc88 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.418833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.419882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.420369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.420826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.421810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:24.422418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.424395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.425801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.426691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.427062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.430240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.430240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:24.430240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.430240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.433988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 373/717] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:24.435023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.452814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 345/718] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:24.454022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.550831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.567781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7365b4-f91f-4902-8bd7-4bf2103e7251 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.580768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.586052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c466d-3580-4bed-a111-9f30ebf4d9a0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.614219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.615378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.615847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.616306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.624580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.627048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:24.645330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:24.646327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:24.647246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:24.650330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 346/719] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:24.656291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.658183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.659064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.659351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.659629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.681749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:24.691897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:24.693071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:24.694048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 374/720] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:24.702045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.777414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3aac5-a674-4e31-8214-160a9cd182aa None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.816948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.818071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.818579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.819523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.821081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:24.821620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.823271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.838382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 347/721] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:24.845087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.859749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.967806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50e84e-1965-4038-ad24-4d6736018642 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.983502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1ec9a-c394-426f-b7ca-9fe84bc62ab1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.024379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.026108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.026607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.027333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.028689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:25.029457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.031342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2af32-c007-428f-8f55-adfdd961f841 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.032935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.033594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.033878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.034154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.035159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:25.035392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.036799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862927c-1400-450f-80e4-453122dceb12 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.048361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 348/722] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:25.048361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e7a29-eb34-4f06-8b8d-2526d99d8a8d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.048361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e7a29-eb34-4f06-8b8d-2526d99d8a8d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.048361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e7a29-eb34-4f06-8b8d-2526d99d8a8d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.048361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 349/723] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:25.049344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 375/724] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:25.053919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e7a29-eb34-4f06-8b8d-2526d99d8a8d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.065924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.067026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.067520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.099000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:25.100225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:25.124619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:25.193047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:25.193457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 376/725] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 14:21:25.216990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.274634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e7a29-eb34-4f06-8b8d-2526d99d8a8d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.341409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.342208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.342488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.342761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.362405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.363861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.388585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65675b2f-e8ed-4f76-98c8-fb4132b9850f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:25.399830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:25.400850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:25.401808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:25.401808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:25.401808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:25.401808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:25.403165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 350/726] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:25.408085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.452082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.452879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.453182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.453451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.475195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:25.485958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:25.486919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:25.487778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 377/727] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:25.497687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.528331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:25.603173 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f256aa-0f62-46e3-b7f9-7b10bb2caec5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.625513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:25.647928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30782d8f-00be-4b86-b1f0-166c917d22af None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.661886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.662568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.662851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.663124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.663862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:25.664165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.666166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.684146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.684821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.685103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.685409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.686162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:25.686514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.689017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.694899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 351/728] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:25.700000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.709253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 378/729] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:25.710765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.784976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:25.815810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc0cd-6fb2-451d-8f46-8647be0907b5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.840157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50296c3b-91d2-47fa-a9ad-626113ae1299 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:25.853766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.854660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.854948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.855225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.866927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.868566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.877356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:25.878421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.878958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.879433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-4c028dcb-b53c-469b-9989-420ca52267ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:25.887470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:25.888396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:25.889345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:25.889345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:25.889345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:25.889345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:25.891336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:25.892979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:25.893257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 379/730] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:25.894595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3662-d02d-431b-898b-e90c7afd2684 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.894884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3662-d02d-431b-898b-e90c7afd2684 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.895159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3662-d02d-431b-898b-e90c7afd2684 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.901609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 380/731] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:25.901609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb3662-d02d-431b-898b-e90c7afd2684 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-05dfb35d-bf46-41e3-8fff-a75989b6fb07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:25.915296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:25.916303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:25.917318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:25.919951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 352/732] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:25.921367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:25.921877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:25.922349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:25.939340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:25.940936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:25.976446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.009967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb3662-d02d-431b-898b-e90c7afd2684 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.016169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 353/733] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:26.022048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.035567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.047390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.048252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.051647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.051647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.051647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:26.051647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.051647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.068828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 381/734] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:26.070002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.142633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28829d-2374-42e5-9760-72e0b39ed7e1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.174931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4868ff-3d48-4600-bcd7-e6d0cab2868f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.179514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.180300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.180653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.181001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.181942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:26.182348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.183924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.203501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 354/735] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:26.204527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.214228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.215568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.216134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.216586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.217603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.219141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.235358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42042823-a495-48b6-91e2-8a615158fdad None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.323202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 382/736] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:26 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:26.323202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.387990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.389091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.389419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.389698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.410105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.411602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.432036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:26.458423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:26.459464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:26.461218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 355/737] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:26.466995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.558272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e7bb-5193-45d2-b092-98f1a1f91d4c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.598028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.598683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.598966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.599243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.612702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.634860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9fa2-a56b-4169-adf7-4316869b92b0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.677983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.678256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.679648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.680371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f95523-748f-429f-91f6-c0c632326d54 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.681146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.681882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.683956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:26.685503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.687139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.709004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 356/738] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:26.714384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.736984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.819156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 383/739] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:26.820509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd119-d757-487b-99b7-8204d376558b None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.820807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd119-d757-487b-99b7-8204d376558b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.821088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd119-d757-487b-99b7-8204d376558b None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.822657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 384/740] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:26.823906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cd119-d757-487b-99b7-8204d376558b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.870636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552bc7b-52b8-40e8-88bc-e8f4550ee18b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.916979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.917661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.918474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.918765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.932066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cd119-d757-487b-99b7-8204d376558b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.936043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.975635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.977274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.977700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.978149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.989440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:26.991197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:26.994636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.007197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.010851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-589ec908-9f3d-4517-8cc6-05e36f74a94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:27.010954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:27.012052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:27.012965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:27.012965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:27.012965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:27.012965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:27.015110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 385/741] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:27.016566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.017028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.017495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.034472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:27.036123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:27.107993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 386/742] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:21:27.108915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.125552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 357/743] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:21:27.133621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.178918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.220204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4c46-2bb0-4ac1-94bf-4386facf938e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.241007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683df75-9f3f-459a-93a6-332ebde52d96 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.247831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.258066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.258999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.259480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.259986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.260852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.261352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.263286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.277287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 387/744] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.278424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.283024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.283581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.283922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.284203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.297596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.299079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:27.317129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:27.318268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:27.319328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:27.319328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:27.319328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:27.319328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:27.319328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 358/745] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:27.319617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.393586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f684-1a42-4ca6-99d7-3f7cdf4e0c87 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.417471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.424793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea98067-8645-4f24-b427-37cb152053f1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.433307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.433975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.434257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.434530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.435247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.436604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.463753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.464591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.464951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.465234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.471763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.473265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f03f831c-e327-4658-8fb4-0a7532b1366e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:27.486995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:27.487892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:27.488824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 359/746] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:27.489666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd863e-bfa2-432b-86fb-b5db5bded629 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.489738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd863e-bfa2-432b-86fb-b5db5bded629 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.490014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd863e-bfa2-432b-86fb-b5db5bded629 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.492978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 360/747] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:27.492978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd863e-bfa2-432b-86fb-b5db5bded629 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.509753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 388/748] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:27 2022] POST /identity/v3/services => generated 240 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:27.510838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.584077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.631979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd863e-bfa2-432b-86fb-b5db5bded629 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.662636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd475-8b75-4a6c-9602-cc3cbe0bf4ee None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.675298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.676314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.676790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.677286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.678198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.678711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.680287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ee9f7-b8e6-43d1-8b8f-7810234a39dd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.701657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 361/749] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.701657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.701657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.701657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.717541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.718210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.718440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:27.718547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.718753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.719550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.719854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.720027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:27.721919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.723087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.734623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 389/750] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.735472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.804085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 362/751] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:27.805365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.877424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eb547-dff9-4874-8a20-5457085246a5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.888617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.926190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.926909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.927193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.927470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.928748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5a16-c69c-4581-a238-801c7df31e44 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.936045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.937544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:27.956762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:27.957926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:27.958822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:27.961242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 390/752] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:27.962309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.975082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.976405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.976920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.977683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.978987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=eceac3a84f1d4eef8fecde82551fb0d1) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.979513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.981969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.997241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 363/753] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/role_assignments?user.id=eceac3a84f1d4eef8fecde82551fb0d1&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.998539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.015284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:28.090142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b78e-80b5-4252-ad05-3accbf009b26 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:28.131134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:28.132095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.133606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.133992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.147230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:28.160323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:28.161376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:28.162257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:28.167069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 391/754] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:28.188738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.238141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4dc40-36b2-4f05-ad34-750cfda33768 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:28.249513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:28.278878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:28.279601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.279887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.280164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.281426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:28.281796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:28.283235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:28.298973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 364/755] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:28.305446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.376511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:28.421454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89febcf-4430-4af7-83e2-55019812ed7a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:28.461043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:28.462231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.462517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.462796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.463636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=59997304919b432184f842b10ab9bba8) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:28.463948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:28.465392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:28.469020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b6ec-2e37-44a8-93b0-7af4760fa785 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:28.472183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 365/756] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:28 2022] GET /identity/v3/role_assignments?user.id=59997304919b432184f842b10ab9bba8&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:28.545947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:28.546642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.546931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.547213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.547975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:28.548288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:28.549701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c8a00-d4a6-4e7e-a819-f6efaa349352 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:28.566585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.567079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 392/757] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:28.694917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feaef60-dd2a-41ec-b8af-a98f7edcb363 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:28.722878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:28.728760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2266c7-2dab-47f9-902d-7ae0ab424e64 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.729055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2266c7-2dab-47f9-902d-7ae0ab424e64 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.729422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2266c7-2dab-47f9-902d-7ae0ab424e64 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.730896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 393/758] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:28.735832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.737342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.737453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:28.737569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.737658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:28.737658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:28.737658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:28.751374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:28.752843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:28.756974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:28.758239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:28.784633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:28.785653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:28.786587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:28.786587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:28.786587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:28.786587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:28.786587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 366/759] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:28.791797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.865617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 394/760] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:28.888151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:28.907865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:28.992976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f3677-4c17-4604-ad77-2c3fb59cdeaf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.003038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426cc3c-c90c-4cff-b0d5-54ae0cb37180 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.030766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.031943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.032442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.032938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.033821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.035379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:29.047930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:29.048877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:29.050100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:29.052709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 395/761] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Jul 12 14:21:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:29.058700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.086965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.087643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.087926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.088202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.088932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:29.089272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.090635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.119702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 367/762] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.124837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.159615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.178807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2100-9aa0-40c8-9744-7915725aa739 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.214119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.215172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.215819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.217189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.218121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:29.218639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.225706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 396/763] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.231406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.232932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.337408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85655500-fe6d-419c-ada5-e4e91165ec47 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.373594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.374651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.375151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.375663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.376576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:29.377102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.378694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.384884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 397/764] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.389250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaeaed-aa4e-4396-bbe1-a67b9fa542df None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.391607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.405611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.482560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.483379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.483663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.483937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.500136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.501564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.510060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959247d5-fc11-4db0-b43b-ff4eaf4c77a6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:29.529556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:29.530616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:29.531579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 368/765] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:29.536519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.549821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.550789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.551283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.551788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.552796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] RBAC: Adding query filter params (interface=public, service_id=7a10ca59a75044b69a2154e11c12bf32, region_id=RegionOne) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:29.553426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.555538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.564719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 398/766] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/endpoints?service_id=7a10ca59a75044b69a2154e11c12bf32&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.570963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.649857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.676868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ac73f-09a9-4cbb-ba35-6e67dfed18a2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.728836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.732554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf67234-a899-4263-a3cf-29fdd84d2d37 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.765888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.767504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.768159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.769500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.770885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.772958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.776285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.777203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.777692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.779095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.780174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:29.781792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:29.783684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63859eb8-e24b-4feb-b7ea-71d51665ec44 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:29.786471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 399/767] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.813727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 369/768] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:29.819605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:29.854370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3cbfc-db9a-46c4-9535-1d840df3ef10 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.854789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3cbfc-db9a-46c4-9535-1d840df3ef10 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.854965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3cbfc-db9a-46c4-9535-1d840df3ef10 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.856587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 370/769] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:29.861514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.861804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.862150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.891680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:29.893046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:29.946354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02858b77-3d66-44b0-9861-76b800066606 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:29.984487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:29.985684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.986182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.987791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.999415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.001632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.002044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 371/770] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:30.003253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:30.021991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:30.023068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:30.024225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:30.024225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:30.024225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:30.024225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:30.025617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.025829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 400/771] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:30.034192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.126154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.139743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0021-9546-4c50-9ad1-a03684a52d71 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.146096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513bd84-7904-4ef6-ab86-a70a01a8b61f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.177872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.178731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.178951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.179232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] PATH_INFO: `/v3/services/network` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.180017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.181409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.183687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.184584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.185072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.185601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.186528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.187039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.188580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:30.193653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:30.194659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:30.195561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 372/772] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:30.204834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 401/773] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.206220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.210063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.302056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.355628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4898-2f1e-459b-85dd-fa3f11a51176 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.391772 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.392568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.392854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.393129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.401086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.402567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:30.418089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:30.419391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:30.420310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 402/774] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:30.425233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.433129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9018ce-2a29-4b20-8e6b-54e87b9ae569 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.515164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.515762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.516046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.516424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.517147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] RBAC: Adding query filter params (name=network) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.517477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.518962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.533118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 373/775] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.538678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.540532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccca78-7438-414b-beff-c14b9c47ef3a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.552000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.567527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.578235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.578953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.579242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.579518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.580368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.580679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.582141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.597851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 403/776] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:30.604377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.713820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d20c50-006d-4839-8884-7d97aaba8644 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.721248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35909c47-daf4-4dad-a01f-272dd441908c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.759057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.760060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.760410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.760755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.763110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=a6b4ed458d984a51829114a73534fce2, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.763642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.764484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.764769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9f32-c12a-4270-b200-d732927a918d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.765077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.765624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.766522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] RBAC: Adding query filter params (type=network) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.767013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.769348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d673f-fd65-43a1-92dd-b1f806508d64 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.776332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 374/777] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.777479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a1a-27cc-4eef-9190-8934f9a1b847 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.777771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a1a-27cc-4eef-9190-8934f9a1b847 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.778051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a1a-27cc-4eef-9190-8934f9a1b847 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.779477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 375/778] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:30.783529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92a1a-27cc-4eef-9190-8934f9a1b847 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.854071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 404/779] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:30 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:21:30.863959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cb2d-c64b-42ee-b8d3-83439d67226f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.864317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cb2d-c64b-42ee-b8d3-83439d67226f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.864610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cb2d-c64b-42ee-b8d3-83439d67226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.881889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.902033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3b02cb2d-c64b-42ee-b8d3-83439d67226f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:30.904013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3b02cb2d-c64b-42ee-b8d3-83439d67226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:30.918908 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92a1a-27cc-4eef-9190-8934f9a1b847 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.953947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.971212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.972546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.973009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.973813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.975088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] RBAC: Adding query filter params (interface=public, service_id=def3b75ef9c14feebefa80b0850d080e, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.975586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.977860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c935-a40b-480b-889c-5ce45883b138 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.991915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 376/780] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/endpoints?service_id=def3b75ef9c14feebefa80b0850d080e&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.991915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b09-fd1f-4c0f-ad21-2936dc74350b None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.991915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b09-fd1f-4c0f-ad21-2936dc74350b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.991915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6b09-fd1f-4c0f-ad21-2936dc74350b None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.996064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 377/781] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:30.996930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6b09-fd1f-4c0f-ad21-2936dc74350b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.011463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 405/782] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:31.013513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.014007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.014510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.033123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:31.034704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:31.110123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 406/783] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:31.111338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.143056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6b09-fd1f-4c0f-ad21-2936dc74350b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.191798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.192442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.192730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.193013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.193722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:31.195122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:31.206959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 378/784] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:31.206959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.227480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:31.235184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:31.254947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e79ee-6f5a-4666-8c66-f542c32255ac None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.330067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.331873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.332639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.333136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.357740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:31.369611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:31.370647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:31.371508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:31.374737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 407/785] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:31.383517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.387905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc39e-290a-462a-9786-f9ff6c935013 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.470715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.471539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.471833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.472113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.486387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:31.501409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:31.502369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:31.503242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:31.504104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 379/786] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:31.509825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.517929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ea99-76df-461a-b98c-360516783b0e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.564962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.566024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.566488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.566955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.568906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:31.569473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:31.571069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:31.580879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 408/787] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:31.597274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.609795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:31.636089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:31.650961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e574-9a5b-45a5-8e1b-38271d290ad5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.688726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.689432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.689718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.690001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.690765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:31.691078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:31.692456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:31.702170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 380/788] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:31.707648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.799151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1c79-9bbc-41a2-a327-a92a0bb3ca74 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.822997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa9975-252f-4183-ba15-e2adc707dcf4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:31.853604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:31.858656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.859459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.859746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.860511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.872445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:31.873929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:31.893004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:31.893994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:31.895679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:31.895679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:31.895679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:31.895679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:31.895679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 381/789] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:31.900382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:31.957880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:31.958705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:31.958989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:31.959263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:31.995285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:31.996746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:32.035095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:32.037232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:32.039941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:32.039941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:32.039941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:32.039941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:32.040450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207702a-1722-4db3-b6aa-87e64a46200a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.043070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 409/790] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:32.047729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.113723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.114325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.114614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.114895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.115634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:32.115944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.125454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.140681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.159776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 382/791] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:32.165460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.181904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c313c51-18fe-4b76-8627-a5025e26b09b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.219267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.220209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.220495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.220773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.222584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:32.222584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.223872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.237260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 410/792] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:32.242310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.268683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.362988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e07a7e-f39a-4756-b671-75d397200ed6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.363997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3594-63a9-4570-b578-9c7e41255545 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.400419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.401542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.401929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.402305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.411111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.412705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-0252e490-4cc0-448f-ad28-512e0735dc5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:32.429207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:32.431029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:32.432217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:32.434201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 383/793] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:32.435727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd83b-1c48-4d00-bd3f-2b8bbc8e2f03 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.436396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd83b-1c48-4d00-bd3f-2b8bbc8e2f03 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.436777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cd83b-1c48-4d00-bd3f-2b8bbc8e2f03 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.442485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 384/794] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:32.442485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cd83b-1c48-4d00-bd3f-2b8bbc8e2f03 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.534538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.545977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.546944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.547300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.547648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.556251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.557824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.563720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cd83b-1c48-4d00-bd3f-2b8bbc8e2f03 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.566752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-0cf2618e-71f8-47e0-90ca-55822e4e7b6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:32.576550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:32.589668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 411/795] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.590629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.591556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.597988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:32.599300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:32.611492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.612737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.613280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.613770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.615116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:32.615734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.618695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.644879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 385/796] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:32.646061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.685179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 412/797] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:32.685179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.776845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203719f2-448b-4402-a563-10f0ea8de70c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.815198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.816171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.816464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.816744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.818092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:32.818407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.820407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.825397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1dbb-0956-4db5-a09e-4f8981dec495 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:32.843124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.843424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 386/798] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:32.845283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.869251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:32.870294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.871507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.871997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.872449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.891662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:32.894305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:32.916266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:32.917238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:32.918163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:32.918163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:32.918163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:32.918163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:32.922214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 413/799] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:32.923083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.968056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f627e77-ce5e-4010-81dd-46a25fe9135a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.006336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.007287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.007674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.008046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.008834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.010408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:33.023883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:33.025996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:33.028005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 387/800] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:33.030066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.030211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fc83-1508-4c26-9ac9-6e6c4535680f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.064393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.065259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.065558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.065834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.077992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.079511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:33.097005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:33.099106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:33.101806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:33.101806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:33.101806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:33.101806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:33.109587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 414/801] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:33.109587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.115417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.125321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.136247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4daf7-b750-494a-8bc3-b156f7d10116 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.173130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.173853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.174140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.174421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.175212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.175654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.177038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.189912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 388/802] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.190817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.221715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93b5ad-7c83-4d63-b631-b3bf6e892159 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.265171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.266375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.266687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.267241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.267958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.268412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.270403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.276644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 415/803] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.277776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.317780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502b9e-534c-46f9-bb3c-47c786fe853a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.354393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.355060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.355346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.355630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.356360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.356665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.358474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.365449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.371034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 389/804] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.371839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.378418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.384322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9a3ca-a20c-4217-a2a2-a84dd63d913a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.420474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.421292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.421577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.421851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.428665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.431895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d3e808ca-9a8a-4c5d-8561-84cddcaaec3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:33.444565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:33.445571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:33.446514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 416/805] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:33.447018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a49b9-d69d-4605-8283-cabd09db1f70 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.447872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a49b9-d69d-4605-8283-cabd09db1f70 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.448113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a49b9-d69d-4605-8283-cabd09db1f70 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.449592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 417/806] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:33.450293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a49b9-d69d-4605-8283-cabd09db1f70 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.502376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b4493-d501-462e-9cf7-359340318c09 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.538066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.538751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.539043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.539319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.540029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.540331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.541729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4cb20-a86e-4356-b980-8e110ac9657a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.551082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 390/807] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.551082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.551082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.551082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.565449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:33.566692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:33.571893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a49b9-d69d-4605-8283-cabd09db1f70 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.605976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.606764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.607048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.607323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.613883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.615309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.625802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:33.629901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:33.630984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 418/808] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:33.631933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.635122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.637892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 391/809] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:33.638689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.730938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ff15e-912e-4484-a4e7-124c43cf8f44 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.739906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2006c-a733-4cd5-b033-c5afd5f691d2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.764602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.765326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.765615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.765888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.766666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.768390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3c6db-3d78-47a4-98a0-79bfbd50756d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.773172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.773830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.774115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.774393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.775126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.776821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.779550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 419/810] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.780756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741c29a-03db-4fcd-a829-328f5b210cef None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.781044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741c29a-03db-4fcd-a829-328f5b210cef None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.781353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741c29a-03db-4fcd-a829-328f5b210cef None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.782575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 420/811] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:33.783257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741c29a-03db-4fcd-a829-328f5b210cef None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.852528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.869209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 392/812] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:33.869209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.908059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741c29a-03db-4fcd-a829-328f5b210cef None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.920727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.948605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.949311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.949599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.949880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.950692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.951002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.952461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.970853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 421/813] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.970853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.981622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aef2a-4777-45aa-80f3-fd1c3205a1a3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.020833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.021666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.021947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.022219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.034805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c77ee969-d68e-46dd-9e5f-14dbb53525e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:34.043041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:34.043998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:34.044866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 393/814] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:34.046474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.046577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.046854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.065366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:34.066596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:34.087247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d784290-ca6e-46f7-aaef-6497a942ac28 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.106467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.123574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.124462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.124742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.125010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/59997304919b432184f842b10ab9bba8/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.127051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=59997304919b432184f842b10ab9bba8, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.128469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.137541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 394/815] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:34.138367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.181992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.200612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 422/816] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:33 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/59997304919b432184f842b10ab9bba8/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:34.201481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.258054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c7452-9bf3-432a-b068-779ec46e8da2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.307926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.308929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.309214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.309550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/eceac3a84f1d4eef8fecde82551fb0d1/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.327099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d51ba-7b97-4c86-8fdc-97cb32f72255 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.327326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=eceac3a84f1d4eef8fecde82551fb0d1, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.328782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.368253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.369023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.369390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.369715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.370529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:34.370837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.372235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.381643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 423/817] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:34.383200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.401965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 395/818] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:34 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/eceac3a84f1d4eef8fecde82551fb0d1/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:34.403057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.464452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.524024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.534181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef8149-e492-4ba6-a707-96b912dae664 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.581691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdd6c4-f3a9-4a8c-b54b-d68777a72c87 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.588241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.589768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.590261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.590738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.604933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.610811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.626021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.626982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.627339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.627689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] PATH_INFO: `/v3/services/network` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.628445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.629955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:34.630667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:34.631536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:34.632994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:34.632994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:34.632994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:34.632994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:34.635475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 396/819] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:34.641654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:34.652454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:34.653759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:21:34.654683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:34.655528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 424/820] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:34.661342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.763712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221700a-9ba3-4b24-90bb-7d94dafd7731 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.778705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d5db0-1328-419b-9fc2-98aec3a7dc74 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:34.781515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.799340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.800015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.800346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.800635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.801484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:34.801797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.803170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.815933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 397/821] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:34.822339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.823339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:34.826387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:34.826677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:34.826964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:34.829450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] RBAC: Adding query filter params (name=network) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:34.829826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:34.830144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:34.832401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:34.845179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 425/822] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:34.853072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:34.987552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384e54-5271-48d3-a331-15a6dfb10b74 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:35.034622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:35.035403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:35.035694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:35.035974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:35.044753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:35.046258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:35.052031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af584d73-6721-4bbe-b1e2-9b0e34cb2f94 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:35.061606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:35.062578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:35.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 398/823] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:35.074191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:35.099863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:35.100549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:35.100829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:35.101640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:35.102393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] RBAC: Adding query filter params (type=network) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:35.102700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:35.104085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:35.118468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 426/824] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:35.123791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:35.198112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:35.229069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde54f78-71f6-4999-a1da-cccfd56d8ebf None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:35.241510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:35.284229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:35.284962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:35.285217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:35.285503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:35.286356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:35.286670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:35.288117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:35.328664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 399/825] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:21:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:35.338835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:35.357190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a22b9-9dfb-4839-a9e9-e9450647212a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:35.387687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:35.388676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:35.388997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:35.389322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:35.390067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:35.391522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:35.485473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efff7b9-4a8c-4ca4-8304-b12efdc3f6f2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:35.504017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 427/826] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:35.520668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:35.542685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:35.543313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:35.543605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:35.543867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:35.622128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:35.623719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:35.875159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 400/827] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:35.880457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:35.991073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.009365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bdd63-faf6-4cff-ad57-4c25facd3a38 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.056139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.057025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.057343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.057623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.072610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.074103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:36.095478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:36.096427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:36.097366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:36.097366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:36.097366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:36.097366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:36.097366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 428/828] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:36.102250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.133668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.215020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13270-35cd-4559-b292-0dd0a735cb4b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.250016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.251030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.251511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.252001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.253048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:36.253585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.255143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496d83-e078-4d9c-8d54-f31dc35ab35b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.268139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 401/829] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:36.273293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.286242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d4bd-bd10-4a99-9cc4-e79e5de10f30 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.286560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d4bd-bd10-4a99-9cc4-e79e5de10f30 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.286803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032d4bd-bd10-4a99-9cc4-e79e5de10f30 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.288551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 402/830] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:36.293315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.293606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.293886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.311273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:36.312457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:36.384778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff38d0d-7211-4ce9-a6b6-f40b3426ec2a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.388552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 403/831] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:36.405074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.417987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.422890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.424297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.424788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.425321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.433612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.435375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.445032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:36.457789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:36.458897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:36.459788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:36.461142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 429/832] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:36.466009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.531633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d9f86-644c-4655-98b9-f13583b41a63 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.570736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.571527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.571806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.572310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.572723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.574107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-21487628-a6ff-41b4-a899-ae3f31b7bf83 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:36.586706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:36.587762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:36.588636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:36.589456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b67ac-4236-44f5-8d87-99c113791585 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.589535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 404/833] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:36.589535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2130-011a-46ac-820a-f200c61184e7 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.590346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2130-011a-46ac-820a-f200c61184e7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.590346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d2130-011a-46ac-820a-f200c61184e7 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.591729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 405/834] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:36.593981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d2130-011a-46ac-820a-f200c61184e7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.624838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.625556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.625841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.626119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.626904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:36.627216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.628646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5508ad-15d2-4316-87ae-a3e96a04b7cd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.640971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 430/835] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:36.642153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.642445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.642726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.661648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:36.662844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:36.679963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.711399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d2130-011a-46ac-820a-f200c61184e7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.725940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.737348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 431/836] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:36.738159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.761938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.762610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.762892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.763167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.763885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:36.764190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.765773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.770436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 406/837] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:36.771209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.855174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382560d-edbd-4b40-9a44-2a8287bc6902 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.877089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03893095-d1d8-467c-9979-ed333e9a0d18 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:36.891768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.892707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.892993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.893292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.894127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:36.894439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:36.895859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:36.903406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 432/838] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:36.903406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.913416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:36.914252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.914531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.914800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.917858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:36.926836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1f2407ef-ff47-4620-8f18-141057219a01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:36.935026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:36.936062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:36.937566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 407/839] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:36.938314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b8e08-448d-42d6-bb49-471159e0e0a9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:36.938314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b8e08-448d-42d6-bb49-471159e0e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:36.938649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b8e08-448d-42d6-bb49-471159e0e0a9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:36.944664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 408/840] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:36.944664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b8e08-448d-42d6-bb49-471159e0e0a9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:36.981942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.034397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebb317-8233-45a4-9094-d7159eacdec8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.059675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b8e08-448d-42d6-bb49-471159e0e0a9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.073330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.074290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.074793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.075167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.075895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.076207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.077623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606e280-cfc3-41c9-b89e-fb158e48e00e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.089249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 433/841] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:37.090777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.091174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.091350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.099380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.100079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.100364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.100645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.101572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.101952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.104311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.113440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:37.114701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 409/842] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:37.116540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:37.121876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.193691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.203204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 434/843] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:37.226684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.250231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427f500-d2e5-41e6-9e6c-d980038775d0 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.291866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.292662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.292944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.293255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.303867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.305710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.307450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:37.323630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:37.324599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:37.325529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:37.325529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:37.325529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:37.325529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:37.325529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 410/844] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:37.330709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.371448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43349b8-094d-46f7-81e8-65e532de48c6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.407644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.408851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.409380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.409857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.422125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:37.434161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:37.435570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:21:37.436466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:37.440934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 435/845] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:37.444813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.462172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd56b18-a7e8-44ff-ba5a-209e47e1cbfa None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.477726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.509227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.509948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.510237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.510512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.512045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.512355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.514024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.533317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 411/846] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:37.535302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.570602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c5e71-b051-4064-9933-0918986a8cf0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.597547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.606734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.607687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.608141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.608585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.612140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.612140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.612140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.621576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 436/847] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:37.626814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.647073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0139-7979-4bd8-b258-1c9a89cab913 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.686755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.688502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.689364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.689848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.700694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.703097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:37.722166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:37.723151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:21:37.724130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:37.727546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 412/848] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:37.736374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.751036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dab82-18e1-4183-8bff-1f5b7ff5fec6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.787599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.788498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.789364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.789527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.789625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.800537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.802057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:37.820351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:37.821391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:37.822307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:37.822307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:37.822307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:37.822307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:37.825517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 437/849] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:37.827740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.854547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:37.862476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286bef4-7721-4f07-b057-8657f9ad04c1 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.909917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.910943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.911784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.912249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.914253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.914771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.916943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:37.938663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 413/850] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:37.942871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da513899-dcb2-43ae-abd6-05f2b08789ad None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:37.943861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:37.986895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:37.987593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:37.987838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:37.988118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:37.988837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:37.989145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:37.990524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.008540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 438/851] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:38.009434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.058537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.075601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc0a55-7755-47dd-b31b-f521b3beeb1c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.103736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.121278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.122004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.122574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.122864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.127677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2126a-f6a8-4b2b-a97f-6e9c463da407 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.168169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.169430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.169960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.170487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.180090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.181948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.197458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:38.198362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:38.199304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:38.200300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:38.200875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.207346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 439/852] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:38.208417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.289250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.306548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 414/853] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:38.308768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.327302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b858d23-3fc1-4274-907d-0519b1d015c0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.367544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.368524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.369131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.369668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.370758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:38.371272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.372902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.388432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.397417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 440/854] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:38.399458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.409137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd0e9-735f-4930-9df1-8db45ed18f1b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.444083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.444911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.445214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.445493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.456482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.458460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:38.476602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:38.477641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:38.478590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:38.478590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:38.478590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:38.478590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:38.482502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 415/855] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:38.483751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.510170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460f50-c29d-44d2-96ee-40cbe148b596 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.526744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.546849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.547573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.547972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.548361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.589343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9803-d66e-4598-8116-a12f2f1e4c5c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.617902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.619499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.624829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.625698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.626146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.626561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.627419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:38.627864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.628740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.629547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.646917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 416/856] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:38.648885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.766754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 441/857] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:21:38.770315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.796356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.801141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1d87-ebc1-49d2-b20c-0324be7312a6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.869882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.870936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.871364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.871778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.875976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:38.879858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3536422-8670-45b2-aab7-039d851e3d3c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:38.884850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.888962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:38.909662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:38.910664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:38.911600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:38.919420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 417/858] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:38.920548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:38.922002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:38.923048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:38.923531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:38.924005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:38.935115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:38.936850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:38.955518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:38.956451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:38.957771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:38.957771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:38.957771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:38.957771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:38.964817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 442/859] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:38.966031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.074286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070cecb-d627-44f8-8a0a-ad8f3fb52e53 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.103561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbf84-c7d5-4793-9d84-fd20bccd097e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.121882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.122557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.122843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.123121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.123848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.124156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.125563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bf511-e502-42ab-a035-5eb483115d3d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.144466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 443/860] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.144466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286153c3-2a1a-495e-a9d1-595417723885 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.144466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286153c3-2a1a-495e-a9d1-595417723885 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.144466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286153c3-2a1a-495e-a9d1-595417723885 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.147788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 444/861] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:39.149025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286153c3-2a1a-495e-a9d1-595417723885 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.149729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.150666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.151197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.151681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.152628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.153040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.154647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c256fc5-4b40-448e-88da-1f4d016d9b04 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.168060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.173979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 418/862] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.173979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.173979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.173979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.193004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:39.194448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:39.209889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.267214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 419/863] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:39.268341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.268521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286153c3-2a1a-495e-a9d1-595417723885 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.308283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.309083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.309398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.309679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.316370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.317852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-fd8aee15-30fa-4f98-a46f-d23c3d1fa73b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:39.332834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:39.333859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:39.334770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 445/864] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:39.335295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111633-14d3-4434-b785-0b0acb76a710 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.335564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111633-14d3-4434-b785-0b0acb76a710 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.335854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d111633-14d3-4434-b785-0b0acb76a710 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.337064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 446/865] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:39.337765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111633-14d3-4434-b785-0b0acb76a710 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.370843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19934a-e434-46f5-b061-fa2cbe88a1eb None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.404023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.404676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.404960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.405278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.406111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=59997304919b432184f842b10ab9bba8) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.406424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.407831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.408241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.413674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 420/866] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/role_assignments?user.id=59997304919b432184f842b10ab9bba8&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.421257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.434861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.449443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d111633-14d3-4434-b785-0b0acb76a710 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.486777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.487639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.487928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.488208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.488877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.490315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-265d333b-1c1c-426f-af34-4aa7d0ba79fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:39.503996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:39.505068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:39.512737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 447/867] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:39.513671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.513671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.513671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.529335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:39.530613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:39.538236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8d0d-edfd-44fc-be8b-b9336441c286 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.577100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.577800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.578106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.578367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.579178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.579475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.580911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.594165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 421/868] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.594713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.619043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 448/869] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:39.620380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.676790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.695067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.702217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff907da-34ca-40c3-a8f7-bfb1f6b0456f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.733437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6f66b-7dca-436d-bc07-afb37aa5d493 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.738339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.739018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.739300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.739575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.740284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.740591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.742737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.752287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 422/870] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.752287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.774587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.775256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.775537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.775811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.776637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=eceac3a84f1d4eef8fecde82551fb0d1) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.776946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.778615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.784929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 449/871] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/role_assignments?user.id=eceac3a84f1d4eef8fecde82551fb0d1&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:39.785757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:39.872058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a2e10-68f0-4732-b905-43a1510f233e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.906123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467aeab4-fd46-4625-8240-0cd3deb28b02 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:39.923409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.924491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.924982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.925497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.926524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.928218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed2b77-95ed-4ac7-afb5-d7b71e8808fb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.940703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:39.946927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:39.947596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:39.947883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:39.948161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:39.958982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:39.958982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:39.958982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd8fff-7324-42be-aad1-bf77f58198d7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:39.958982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 450/872] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:40.015438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:40.021729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 423/873] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:39 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:41.533909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be855f-3a22-428f-99c2-ef02b056adb4 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.534429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be855f-3a22-428f-99c2-ef02b056adb4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.534509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be855f-3a22-428f-99c2-ef02b056adb4 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.536105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 451/874] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:41.541475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.541882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.542046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.559675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:41.560874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:41.637183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 424/875] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:41.658734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be855f-3a22-428f-99c2-ef02b056adb4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:41.767799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be855f-3a22-428f-99c2-ef02b056adb4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:41.802367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:41.803172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.803452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.803725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.804943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:41.806345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-647fe58b-30c6-4f88-893c-5e41d962c399 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:41.818162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:41.819174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:41.820095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 452/876] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:41.825241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:41.928182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebc11e-ef6a-4486-8efb-eba0a4ddaad4 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:41.965225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:41.974403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953e42f-2c04-4056-9cc5-7e96cecbd824 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.974684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953e42f-2c04-4056-9cc5-7e96cecbd824 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.974956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953e42f-2c04-4056-9cc5-7e96cecbd824 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.977007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 453/877] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:41.979045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:41.979906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.980409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.980854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.984101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:41.984221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e493-9d45-4e96-a86a-f877408d25ff None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:41.984305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:41.984379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e493-9d45-4e96-a86a-f877408d25ff None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:41.984379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e493-9d45-4e96-a86a-f877408d25ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:41.984521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:41.990035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 425/878] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:41.998164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.003013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-70e0e493-9d45-4e96-a86a-f877408d25ff None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:42.004198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-70e0e493-9d45-4e96-a86a-f877408d25ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:42.068546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.085538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 454/879] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:42.087596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6439c3-ed60-49cd-a9a4-5e007349c832 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.087948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6439c3-ed60-49cd-a9a4-5e007349c832 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.088279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6439c3-ed60-49cd-a9a4-5e007349c832 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.090372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 455/880] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:42.095091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab6cd-1abc-4097-94ff-79435ac40e18 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.095438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab6cd-1abc-4097-94ff-79435ac40e18 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.096016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab6cd-1abc-4097-94ff-79435ac40e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.103394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e01f-fe31-4056-949e-917aa94e562a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.117339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c14ab6cd-1abc-4097-94ff-79435ac40e18 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:42.118528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c14ab6cd-1abc-4097-94ff-79435ac40e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:42.141342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.142110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.142399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.142671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.143373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:42.143674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.145033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.149867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 426/881] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:42.150623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.189878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 456/882] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:42.191613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350d5f4-d1cb-437b-af94-3246eb0aeaa9 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.191911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350d5f4-d1cb-437b-af94-3246eb0aeaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.192194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350d5f4-d1cb-437b-af94-3246eb0aeaa9 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.193719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 457/883] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:42.199291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.199625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.199878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.218742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:42.219988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:42.222212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.280573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2ca9-c6a8-4bb3-9257-09339410d24e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.300355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 458/884] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:42.301124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.328134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.328873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.329174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.329474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.340829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:42.349073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:42.350046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 427/885] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:42.351907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.363564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.414637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd61d-e777-4e45-a8df-732e2a3e1439 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.459189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.459854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.460761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.460936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.461817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.463222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.471675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaf544-966e-45ce-8629-c9b32a42a96d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.485234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.505518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.506306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.506585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.506857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.508016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.509390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:42.521049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:42.522275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:42.524305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 428/886] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:42.528314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.552383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 459/887] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:42 2022] POST /identity/v3/services => generated 269 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:42.557339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.644968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea60d0-52e5-4168-a241-30ec75bf9ff5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.691705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.692553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.692758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.693041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.693844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:42.694156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.696184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.711290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 429/888] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:42.713316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.719565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db0185-749a-4ce9-906e-d44ffb244bb6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.732441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.759196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:42.765610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.766282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.766598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.766879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.767591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:42.767900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.769375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.775405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 460/889] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:42.781196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.842592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fbf27-d130-4055-9d8e-65bbb7d4eb0c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.878924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.880155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.880666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.881142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.893581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.895410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.912534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ee60f-e0af-488a-b35c-d81956f2b9ee None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:42.914491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:42.915357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:42.916265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:42.916265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:42.916265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:42.916265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:42.918581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 430/890] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:42.925328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:42.960513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:42.961594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:42.961878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:42.962163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:42.963148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:42.963460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:42.964828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:42.973241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 461/891] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:42.976731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.016630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.044213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.066157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc45de-3f5a-4aa9-b59b-a32fc4769fa8 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.088589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55933973-c959-4f09-84f2-28c97ab7bd03 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.106841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.107828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.108317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.108822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.109808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:43.110320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.111875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.129325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.129325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.129325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.129325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.129728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] RBAC: Adding query filter params (interface=public, service_id=39ed491cf90c452a8897278b74560f1e, region_id=RegionOne) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:43.129728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.129728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.130417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 431/892] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:43.131485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.135635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 462/893] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:42 2022] GET /identity/v3/endpoints?service_id=39ed491cf90c452a8897278b74560f1e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:43.142151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.248700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c885656-a82d-4172-90f8-c0fc304e0d71 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.258308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.261746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e782fa1-cfc7-4bb6-a63c-76c3cd8827c6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.300203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.300998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.301308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.301484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.301620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.302419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.302846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.303281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.304666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.306227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.310481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.311898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.313107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 463/894] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:43 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:43.328555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:43.329786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:43.330776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:43.334628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 432/895] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:43.336695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.449183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9419-0d37-437e-8641-3ffb2be054c3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.482344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.483008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.483291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.483565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.484397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:43.484701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.486148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.504860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 464/896] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:43.511245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.520578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.613881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dc6e1-3a5a-47a6-a046-5dc1ec027234 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.648957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.649610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.649960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.650165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.719002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.720460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:43.729315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.828919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 433/897] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:43 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:43.831210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:43.857449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.935327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f0e5-1211-49b2-9338-cdb72fbfd6c1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:43.969936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:43.971000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:43.971485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:43.971940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:43.982764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:43.984398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:44.001750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:44.002766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:44.003693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:44.003693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:44.003693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:44.003693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:44.009251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 465/898] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:44.012195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.112955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd183-60d3-4a10-8ff8-5b0288aeb9d5 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.120745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:44.148637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.149330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.149613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.149887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.150612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:44.157379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.157379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76c514-d7bc-4fc7-a386-91f6cc0e2356 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.168256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 434/899] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:44.169625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.241217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ead58d-31d8-40a8-a12f-07e3e2dd477f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.241553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ead58d-31d8-40a8-a12f-07e3e2dd477f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.241792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ead58d-31d8-40a8-a12f-07e3e2dd477f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.243422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 435/900] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:44.245622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:44.249941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.250887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.251403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.274164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:44.274350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2c238-bf09-499d-94a6-07e0adb3b225 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.275951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:44.310187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.310885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.311167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.311443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.318465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.320439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:44.334206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:44.335714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:44.336632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:44.340704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 466/901] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:44.341648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.377766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 436/902] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:44.401393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.471535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea1e5-7a5e-4ede-a401-7d37edb23a84 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.509605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.510272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.510562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.510844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.511897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:44.512211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.513759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.515737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:44.533472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 467/903] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:44.534214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:44.535126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.542580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fa5ad-bd46-456c-addc-10d59ad50f40 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.590594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.591306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.592352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.592635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.593467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.595025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5eed8-4868-4bc8-b430-505840f4c950 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.656756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab178c6-1d8c-43ab-8b93-8561e3d5feb8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.668858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 437/904] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:44 2022] POST /identity/v3/services => generated 241 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:21:44.671453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c2f87-ed61-43ee-ac10-d9d248a1ecaf None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.671752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c2f87-ed61-43ee-ac10-d9d248a1ecaf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.672028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c2f87-ed61-43ee-ac10-d9d248a1ecaf None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.673716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 438/905] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:44.679787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.680126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.680346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.699017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:44.700178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:44.718505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.720216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.720733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.721272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.722409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] RBAC: Adding query filter params (role.id=3e43f773e9c94aa5b6d013da39189b16, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=eceac3a84f1d4eef8fecde82551fb0d1) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:44.723531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.725812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.735621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 468/906] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/role_assignments?user.id=eceac3a84f1d4eef8fecde82551fb0d1&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3e43f773e9c94aa5b6d013da39189b16 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:44.785553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 439/907] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:44.810085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:44.878329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:44.931584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0f482-f5a2-4185-a5bf-d884c4c91642 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:44.967840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:44.968934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:44.969394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:44.969820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:44.970649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:44.972164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:44.985794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:44.987013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:44.988107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:44.990695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 469/908] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:44.997237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.087288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:45.103098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03738606-43f5-4a8c-80c1-13257e39f6d1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.138142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.138820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.139508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.140155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.140880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:45.141473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.143446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.149657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 440/909] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:45.155361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.269256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955899b-2560-4685-b547-e92773d21cf6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.288033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:45.306735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.323221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41f0b6-bdfe-490d-a8aa-07070b597524 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.323722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 470/910] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:45.323806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.427434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a44b0-cc3c-42d8-a5f5-3d472134ad54 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.463540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e099-74ee-4ed8-89b6-75222ba87e1b None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.463753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.466709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e099-74ee-4ed8-89b6-75222ba87e1b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.466709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e099-74ee-4ed8-89b6-75222ba87e1b None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.466891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.466891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.466891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.466891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] RBAC: Adding query filter params (interface=public, service_id=68bdb423bf054f389e2a53f2a4ef2242, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:45.466891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.467215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.469086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 471/911] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:45.470546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.470833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.471104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.472892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 441/912] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/endpoints?service_id=68bdb423bf054f389e2a53f2a4ef2242&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:45.478823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.489295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:45.490429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:45.494649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:45.571010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 472/913] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:45.580867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b6b91-5153-4a4c-9ac3-482a4263162a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.589896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.615879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.616427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.616710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.616987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.617768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.619123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.625384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 442/914] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:21:45.702474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05649ddb-898c-40e6-8e1c-5f297351b414 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.718362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:45.740982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.742150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.742580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.742999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.743800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.745360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:45.759322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:45.760630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:21:45.761933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:45.769271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 473/915] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:45.771469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:45.859429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:45.882367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1116-bf28-4101-ac16-4fe8a0041574 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:45.928413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:45.929946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:45.930416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:45.930965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:45.932162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:45.932809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:45.935112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:45.945301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 443/916] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:45.948991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:46.054290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c523c-5cb1-4704-98f2-aeaec76972f1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:46.089267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:46.090155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.090582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.091001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.091841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:46.092290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:46.093803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af309de-dbdd-4583-b072-a21adeaf7a70 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:46.103616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 474/917] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Jul 12 14:21:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:46.105083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:46.211329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76ceeb-893f-4b9a-b96f-c60039e81cb7 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:46.246805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:46.247476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.247759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.248033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.248793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:46.250194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc6dc9-3761-4697-aa5a-5dfc40f76cb2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:46.337598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:46.344696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 444/918] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:46.862806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55562c49-eef1-4670-bae4-c974c0b83980 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.863211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55562c49-eef1-4670-bae4-c974c0b83980 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.863399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55562c49-eef1-4670-bae4-c974c0b83980 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.865012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 475/919] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:46.870119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9b25-8804-4774-b84e-76d500888793 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.870411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9b25-8804-4774-b84e-76d500888793 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.870516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202cd2-ceca-4a0d-89cf-2c9e61098417 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.870916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a9b25-8804-4774-b84e-76d500888793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.870991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202cd2-ceca-4a0d-89cf-2c9e61098417 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.870991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202cd2-ceca-4a0d-89cf-2c9e61098417 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.872564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 445/920] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:46.880771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:46.881065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:46.881400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:46.891357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-044a9b25-8804-4774-b84e-76d500888793 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:46.892561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-044a9b25-8804-4774-b84e-76d500888793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:46.900854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:46.902069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:46.931436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:46.983057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 476/921] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:46.992142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 446/922] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:47.006386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a9b25-8804-4774-b84e-76d500888793 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.015554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.128216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a9b25-8804-4774-b84e-76d500888793 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.138266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f302-cba7-4185-a2f9-62708c82cc30 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.168690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.172216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.173770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.174204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.174775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.176061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.178192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.180686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.181875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.185473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.185473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.193180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:47.193485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:47.195214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:47.196266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:47.203902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 477/923] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:47.204875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:47.205327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:47.206413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:47.207258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:47.212912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 447/924] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:47.213976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.249700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.324709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de9d5-9c9f-4e39-91fa-8d73d930d17c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.325443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9d78-a824-4823-8088-216e637c9423 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.361082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.361551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.362297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.362418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.362810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.363009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.363287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.363582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.364329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:47.364604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:47.364843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.365239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.366496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.366926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.379782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 448/925] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:47.379956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 478/926] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:47.380741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.383614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.431415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.486193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e8bd-7d55-4f5c-9360-9af7ca0ca026 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.498544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.507528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fb8bf-89a4-44c4-8967-bfbb1f5ea53f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.525869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.526478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.526764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.527041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.538280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.539735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.544802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.546357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.546942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.547460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.548388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:47.548963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.550570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.557730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 479/927] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:47.562593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:47.563497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:47.564425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:47.564425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:47.564425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:47.564425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:47.564655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 449/928] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:47.566093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.567217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.669003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.673318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb382dd-142f-4e81-b71c-d1f5ba7a97da None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.673900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dde0a-8fea-45ae-ab6b-3803fb9d6670 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.709992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.710756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.711047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.711337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.712473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] RBAC: Adding query filter params (interface=public, service_id=f28b2cf2d5f546bf987840059cb97b54, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:47.712818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.713567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.714517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.714634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.715152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.715737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.716723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:47.717302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.718954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.725228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 450/929] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/endpoints?service_id=f28b2cf2d5f546bf987840059cb97b54&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:47.727561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.741550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 480/930] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:47.747669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.771017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:47.849275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9b2a3-14a2-4237-853e-bc36ddfc2a4e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.854063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252830f6-d12e-4cdf-9147-8a43db9afa3e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:47.886454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.887158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.887414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.887690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.888541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.889976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c55d6-1c93-473a-a2f7-711fb1a5aac6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.890461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:47.891277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.891566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.891997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.896531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 451/931] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:21:47.898091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67eeff-9a3e-4ee5-bbe4-6a37df964560 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.898385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67eeff-9a3e-4ee5-bbe4-6a37df964560 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.898665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67eeff-9a3e-4ee5-bbe4-6a37df964560 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.899952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 452/932] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:47.901679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:47.903788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:47.905752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:47.906043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:47.906324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:47.925516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:47.926572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:47.927638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:47.928233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:47.928233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:47.933843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 481/933] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:47.939341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:47.965997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.037649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 453/934] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:48.053424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.056580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.082486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8f3c-ca79-405b-9030-b9f49bf6fa8a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.129651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.131333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.132002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.132550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.133850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:48.134740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.136895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.159356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 482/935] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:48.165596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.191854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4c749-bfc3-41df-a404-f41001e7fcd3 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.236924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.238243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.238823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.239295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.240138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.242071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:48.260794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:48.262220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:21:48.263114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:48.266377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 454/936] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:48.271895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.280904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ec126-9cf0-417d-9f5f-87d5dde633e8 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.303312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.327143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.328301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.328589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.328864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.332661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.343117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.345617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:48.369344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:48.370357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:48.371300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:48.371300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:48.371300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:48.371300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.375765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 483/937] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:48.376623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.407846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565a205-4a50-45c3-9eed-f337ba2d6e3d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.455939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.455939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.455939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.455939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.457252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:48.457654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.459029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a0132-ee6a-4425-88ba-6200717a0565 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.464995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 455/938] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:48.466372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a88f9-4738-4831-96a1-55b2cfaee7d1 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.466661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a88f9-4738-4831-96a1-55b2cfaee7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.466935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a88f9-4738-4831-96a1-55b2cfaee7d1 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.468271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 456/939] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:48.473510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.473792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.474072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.493267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:48.494751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:48.495835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e915338-fb73-4e22-b2df-5221c729fe67 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.534433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.535097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.535384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.535658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.536376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:48.536685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.538110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.553840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 484/940] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:48.553840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.570183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 457/941] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:48.571090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.595553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.603547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.655096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc26d2e-8205-4583-b000-12c2e5e24c1f None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.688138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff328a4-7c51-4a5e-920b-9fa0ff943bd7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.690515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.691179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.691462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.691739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.692444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:48.692753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.694691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.699567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 485/942] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:48.700323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.731589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.732711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.733151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.733599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.741121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.742733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:48.756989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:48.757929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:48.758834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:48.764277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 458/943] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:48.764277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.809418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ca0b-f9de-4211-acec-607dc650c13b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.824005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.838531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:48.846315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.846973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.847257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.847536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.848237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.849704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.851092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING keystone.server.flask.common [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 14:21:48.863158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9d64-4693-45a1-a527-6114cf1f8dba None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:48.898423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:48.899090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:48.899367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:48.899638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:48.900369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:48.901811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:48.907355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.sql.core [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:21:48.907355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 14:21:48.907355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [parameters: {'id': '78d495ac34894ee4a06882bfafae7fc8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 14:21:48.907355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80346) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 78d495ac34894ee4a06882bfafae7fc8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 78d495ac34894ee4a06882bfafae7fc8. Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:21:48.923856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application result.read() Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 14:21:48.924905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application yield resource Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise self.value Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application session.commit() Jul 12 14:21:48.926253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:21:48.927241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:21:48.928164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application result.read() Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:21:48.930835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [parameters: {'id': '78d495ac34894ee4a06882bfafae7fc8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.931947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:48.932958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 78d495ac34894ee4a06882bfafae7fc8. Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 486/944] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Jul 12 14:21:48 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:48.934245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:48.977996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 459/945] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:48.979476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:49.048784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eccd9f-cbce-4d6b-b6e9-9791a199a894 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:49.111943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:49.113760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:49.114778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:49.115248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:49.116840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:49.117366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:49.120293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:49.132977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c8051-a675-4d21-8711-28cfc6da2713 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:49.145123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 487/946] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:49.146969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:49.171299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:49.178041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:49.183352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:49.184853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:49.184853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:49.184853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:49.195772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:49.197988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:49.220161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:49.221206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:49.222143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 460/947] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:49.223774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:49.268655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf46cf-b394-4944-920f-5961b5bbd0f2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:49.307759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54173f50-79f4-4124-a259-684835977de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:49.309614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173f50-79f4-4124-a259-684835977de6 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:49.310129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173f50-79f4-4124-a259-684835977de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:49.310618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54173f50-79f4-4124-a259-684835977de6 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/eceac3a84f1d4eef8fecde82551fb0d1/roles/3e43f773e9c94aa5b6d013da39189b16` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:49.313388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173f50-79f4-4124-a259-684835977de6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=eceac3a84f1d4eef8fecde82551fb0d1, role_id=3e43f773e9c94aa5b6d013da39189b16)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:49.315029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54173f50-79f4-4124-a259-684835977de6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:49.347649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb44d9-d81f-4d2a-bf13-fbc4b836c660 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:49.383503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:49.384166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:49.384493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:49.384771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:49.385608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:49.385952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:49.387613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b04baa-eee5-4709-ab95-9331f47f3d38 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:49.388499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 488/948] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:49 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/eceac3a84f1d4eef8fecde82551fb0d1/roles/3e43f773e9c94aa5b6d013da39189b16 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:21:49.417914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 461/949] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:50.026067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ab07-333d-4f64-bc0b-e83f7983ae1d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.026608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ab07-333d-4f64-bc0b-e83f7983ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.027055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ab07-333d-4f64-bc0b-e83f7983ae1d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.028817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 489/950] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:50.034361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.035235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.035735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.053866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:50.055387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:50.102895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:50.125609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 462/951] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:50.150399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240ab07-333d-4f64-bc0b-e83f7983ae1d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:50.244754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:50.262804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240ab07-333d-4f64-bc0b-e83f7983ae1d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:50.298679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:50.299754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.300261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.300853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.301769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:50.305655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:50.316733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:50.318146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:50.319114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:50.321977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 490/952] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:50.328502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:50.436599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9d7ce-deb9-4dbb-bec6-f74b70d05209 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:50.474246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:50.475082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.475976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.476254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.476957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:50.477289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:50.478651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:50.483948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 463/953] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:50.489398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:50.597685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5f68f-0c08-4c84-883a-116134600e69 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:50.635190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:50.635886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.636445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.636736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.637461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:50.637773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:50.639169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b4519-3bcb-4607-aeb0-3ac4e8d83895 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:50.645553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 491/954] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:50.652585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:50.706786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:50.762725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895c2e-3d71-4189-9c5c-33948580bb8b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:50.797186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:50.797862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:50.798143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:50.798415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:50.799157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:50.800746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08977f3a-e856-4b6f-bd53-38c3f5aa480a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:50.880835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 464/955] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:51.125042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76653de5-a925-4d18-8716-7398e685da04 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.125553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76653de5-a925-4d18-8716-7398e685da04 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.125924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76653de5-a925-4d18-8716-7398e685da04 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.127652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 492/956] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:51.133483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.133958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.133958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.152488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:51.153731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:51.229545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 465/957] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:51.251771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76653de5-a925-4d18-8716-7398e685da04 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.268263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:51.364819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76653de5-a925-4d18-8716-7398e685da04 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:51.401110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:51.401862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.402149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.402430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.403088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:51.404462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-1f8086f0-466a-4577-9373-3a056760071e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:51.417230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:51.418306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:51.419240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 493/958] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:51.424826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.455328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:51.523407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034a90-418e-4ef9-8997-b001dea70c4a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.523743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034a90-418e-4ef9-8997-b001dea70c4a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.523988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14034a90-418e-4ef9-8997-b001dea70c4a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.525712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 494/959] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:51.531023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.531303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.531587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.532149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501407-ab38-4c59-b53e-736c28696b1a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:51.549264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:51.550830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:51.574052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:51.574727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.575254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.575541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.576597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:51.577053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:51.578726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:51.585384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 466/960] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:51.590762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.636949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 495/961] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:51.660498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.671007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:51.699143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:51.708809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc425-414e-4c0a-9299-50cedfe9104b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:51.745601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:51.746269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.746556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.746836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.747549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:51.747856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:51.749233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7396c-9bd4-4dcd-b1ea-51c70cadaf3f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:51.753612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 467/962] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:51.754792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4685-eea4-45a0-80d8-72b272e98d4d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.755078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4685-eea4-45a0-80d8-72b272e98d4d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.755356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4685-eea4-45a0-80d8-72b272e98d4d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.756803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 468/963] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:51.762524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.762814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.763096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.780941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:51.782206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:51.783474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b61307-24cc-4c3c-ae29-713e6e21646e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:51.822726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:51.823965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:51.824478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:51.825242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:51.842539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:51.851435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:51.852601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:51.859131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 496/964] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:21:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:51.862187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.881624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 469/965] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:51.900788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:51.950859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.000231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebec1b9-5bef-448c-a5d7-8707411b6a93 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.010981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.030459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eaac12-5673-4833-88ea-6da90b0d07ce None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.061605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.062922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.063651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.064119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.065797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:52.066316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.068811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.075680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.076431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.076788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.077125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.077947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.079449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.080927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING keystone.server.flask.common [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 14:21:52.086430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 497/966] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:21:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:52.090096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.134370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.sql.core [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:21:52.134370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 14:21:52.134370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [parameters: {'user_id': 'b7dcc6396ea64229b52e57bfbde13df9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 14:21:52.134370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80347) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result.read() Jul 12 14:21:52.145733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:21:52.146652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application yield resource Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise self.value Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application session.commit() Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:21:52.148188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:21:52.149267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:21:52.150208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application result.read() Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:21:52.152623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b7dcc6396ea64229b52e57bfbde13df9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:52.154708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 14:21:52.156706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 470/967] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Jul 12 14:21:51 2022] POST /identity/v3/users => generated 160 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:52.159115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.214986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd7e50-06d9-4e53-b70d-3b71cd2e8fbc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.253969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.264951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.266251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.266690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.270742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.278648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7d04a-b00f-4e7b-9851-14dc727f60bd None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.282048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.283487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:52.306281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:52.307442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:52.308514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:52.308514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:52.308514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:52.308514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:52.308514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 498/968] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:52.313646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.324524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.325493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.325947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.326486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.337224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.339355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:52.357616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:52.358861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:52.360078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 471/969] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:52.366575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.376026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.441092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a163-902c-4683-b344-39f73c0dff18 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.478602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.479196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.479489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.479913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.480489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:52.480802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.482219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.489541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf624f-369d-43a3-ad53-7ba25518570d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.499638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 499/970] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:52.500663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.526800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.528230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.528666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.529124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.530679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:52.531236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.532820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.546033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.580678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 472/971] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:52.624717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcc0a3-31d0-4f4d-9053-fc234a1f68a2 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.664029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.664860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.665148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.665460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.673796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.675473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:21:52.692712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:52.694060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:52.695270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 500/972] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:52.700219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.777671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.819108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996d357-09be-4c3a-9179-cf99e0538b6c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.855480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:52.856249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.856600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.856941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.857885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:52.858305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:52.859761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f2009-07b6-483a-b433-48ea97de24b2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:52.878300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 473/973] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:52.883575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:52.969368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e43b6-d7a9-4adf-841d-8034f6852303 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.969759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e43b6-d7a9-4adf-841d-8034f6852303 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.969934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e43b6-d7a9-4adf-841d-8034f6852303 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.971281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 474/974] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:52.976810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:52.977105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:52.977426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:52.986914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:52.990659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380223d-40e5-471b-8285-be94abe4f3d3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:52.993973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:52.995471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:53.027583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.030395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.030395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.030395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.064419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 475/975] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:53.087847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.098487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.099979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.146709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.196553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533e6e5-a108-4a5b-aaaa-f96a558132d3 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.204216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.208648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 501/976] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:21:53.214327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.236349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.237164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.237464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.237739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.238394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.239760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:53.251762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:53.252766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:53.258568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 476/977] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:53.259842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.321540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907a54-3cba-42fc-a141-aca180277cef None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.360276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.361072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.361385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.361857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.364938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491f2b-467e-4da8-a90e-4cbb9e2712ab None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.371782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.373236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.382938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:53.392373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:53.399352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:53.400586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:21:53.400586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:53.400586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:21:53.400586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:53.400586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 502/978] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:53.401270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.404241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.405108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.406268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.406559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.407278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:53.407595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.408961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.415458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 477/979] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:53.420952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.469554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.518503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff25479-fc19-41df-b5f5-638d308d0b96 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.529489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a347c4a-f6aa-4df1-bab8-f4cc1cb6d26c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.555301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.555991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.556273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.556551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.557305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:53.557638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.558973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.569608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.570511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.570860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 503/980] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:53.571149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.571430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.572198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:53.572452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.573863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.576183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.580154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 478/981] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:53.585245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.636530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.681895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c5f6a-ee5f-4cfe-8fe9-115c86a11dd3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.693004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12239a4-d45f-4094-a912-ba8dd980ded8 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.695946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.717936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.718737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.719027 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.719311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.726289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.728360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.729251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.729951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.730395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.730675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.731777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] RBAC: Adding query filter params (interface=admin, service_id=f28b2cf2d5f546bf987840059cb97b54, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:53.732097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.733474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.743535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 479/982] 10.222.0.30 () {56 vars in 1249 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/endpoints?service_id=f28b2cf2d5f546bf987840059cb97b54&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:21:53.743679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:53.744703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:53.746443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 504/983] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:53.750023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.750145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.852897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f1cdb-8e1d-4c40-859a-fcfcac6a1d5e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.859517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7ec1-88ca-4f1a-8b96-8480d26af936 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:53.861895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:53.886456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.887110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.887396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.887668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.888336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.889733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a056de-0456-44fe-b83b-1fdbe3300433 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.893909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:53.894561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.894844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.895120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.895548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 480/984] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:21:53.895905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:53.896213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:53.896829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7660a-fb88-40cc-a396-d3d9e3933856 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.897117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7660a-fb88-40cc-a396-d3d9e3933856 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.897415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7660a-fb88-40cc-a396-d3d9e3933856 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.897843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:53.901212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 481/985] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:53.904494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:53.905545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:53.905823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:53.911203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 505/986] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:53.917796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:53.926451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:53.927751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:53.955897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:54.004936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 482/987] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:54.027905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:54.035479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a1f8-a80d-4811-88e8-2cfbb31568bc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:54.079052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:54.079703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:54.079988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:54.080267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:54.081106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] RBAC: Adding query filter params (role.id=3e43f773e9c94aa5b6d013da39189b16, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=eceac3a84f1d4eef8fecde82551fb0d1) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:54.081482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:54.082912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275ffab-b230-49c5-830a-0b0342e20e23 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:54.089045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 506/988] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:53 2022] GET /identity/v3/role_assignments?user.id=eceac3a84f1d4eef8fecde82551fb0d1&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3e43f773e9c94aa5b6d013da39189b16 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:54.140917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e385-00a7-44c2-9375-97d1b0f2128a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:54.142817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:54.179917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:54.180699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:54.180890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:54.181166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:54.181920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:54.183314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944ba98-bb95-4c40-b056-ce8ae7efe0fe None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:54.251669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 483/989] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:54 2022] POST /identity/v3/services => generated 242 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:21:55.960113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc461-c052-4092-908e-84c5557d7d0f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:55.960657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc461-c052-4092-908e-84c5557d7d0f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:55.960657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc461-c052-4092-908e-84c5557d7d0f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:55.962118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 507/990] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:55.967221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e50dc-5277-4c51-a53e-f8aff3662403 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:55.967617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e50dc-5277-4c51-a53e-f8aff3662403 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:55.967819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e50dc-5277-4c51-a53e-f8aff3662403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:55.985218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f88e50dc-5277-4c51-a53e-f8aff3662403 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:55.986411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f88e50dc-5277-4c51-a53e-f8aff3662403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:56.063320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 484/991] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:56.086570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc461-c052-4092-908e-84c5557d7d0f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.143481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8f2a6-e952-4811-8ec4-d726c52ab52c None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.143815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8f2a6-e952-4811-8ec4-d726c52ab52c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.144055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8f2a6-e952-4811-8ec4-d726c52ab52c None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.145466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 485/992] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:56.150886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.151174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.151458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.167717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:56.168891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:56.185235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.192343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc461-c052-4092-908e-84c5557d7d0f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.228580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.229350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.229648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.229932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.230576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.231950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.236264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 486/993] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:56.243990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:56.245100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:21:56.246294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.249453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 508/994] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:56.251025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.258170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.272434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.356170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1296d-18dc-4a1b-8bb6-9e48192309f2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.357344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78538a-6ab8-4223-bfc9-d927a13cf3e6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.389754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.390138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.390507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.390793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.390919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.391071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.391203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.391477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] PATH_INFO: `/v3/services/image` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.391775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:56.392088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.392213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.393487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d34af-1943-4156-83fa-3a719c167b77 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.393603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.398881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 487/995] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:56.400150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310d99-45e7-47fc-8355-3952d37d0386 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.400438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310d99-45e7-47fc-8355-3952d37d0386 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.400714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310d99-45e7-47fc-8355-3952d37d0386 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.402111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 488/996] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:56.404353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e310d99-45e7-47fc-8355-3952d37d0386 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-db3dbe86-f4bc-447c-ad7d-aa4b99953a0b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:56.406125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:56.407212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:21:56.408594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 509/997] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:56.409573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.409573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.409573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.425811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:56.427003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:56.445842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.488827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.493939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 510/998] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:56.494768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.509265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e310d99-45e7-47fc-8355-3952d37d0386 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.542981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.543748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.544304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.544805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.545932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:56.546245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.547612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.552664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 489/999] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:56.553974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.597645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a1bf-c48d-49f8-a5de-fa136ac8802b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.632053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.632599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.632889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.633188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.633897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] RBAC: Adding query filter params (name=image) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:56.634206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.635569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.640508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 511/1000] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:56.641293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.656027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36430f-b87a-42ac-b195-95a4dbd50c27 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.658717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.691062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.691844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.692118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.692392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.693036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.694984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:56.707136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:56.708137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:56.709000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 490/1001] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:56.710390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.721377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.743282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8994-3a7b-493b-b11f-eabb8320ff3e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.778034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.778688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.778973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.779251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.779984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.781375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.807091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a4b0e-316b-447b-8baa-ef4b7fef5e33 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:56.840299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:56.840962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:56.841278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:56.841558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:56.842269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] RBAC: Adding query filter params (type=image) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:56.842580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:56.843939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:56.845332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 512/1002] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:21:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:56.846301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:56.853225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 491/1003] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:56.947039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:56.978830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa08c-866a-48a3-94ec-4ed3595d700a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:57.017277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:57.018295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:57.018582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:57.018854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:57.019572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:57.019878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:57.021376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:57.028125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 513/1004] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:57.033528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:57.115476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:57.140959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a791-32cf-4514-ae4d-26e9a4857e8f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:57.176545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:57.177229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:57.177578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:57.178355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:57.179072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:57.179379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:57.180740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:57.186142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 492/1005] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:57.192057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:57.295510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d21ff-690d-4847-abef-82b02b2d0598 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:57.330941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:57.331605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:57.331891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:57.332169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:57.332961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] RBAC: Adding query filter params (interface=public, service_id=388e6f446fc040fdaefb1b409d487002, region_id=RegionOne) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:57.333304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:57.334748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9111f-4c18-4bb3-bb68-9ea875c27b89 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:57.340066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 514/1006] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:21:57 2022] GET /identity/v3/endpoints?service_id=388e6f446fc040fdaefb1b409d487002&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:57.345368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:57.453712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278cd96-f632-418d-b708-f90a87a7e976 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:57.488317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:57.488979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:57.489298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:57.489576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:57.490260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:57.492181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368a7e9-7f26-4671-8ece-6defc40fb923 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:57.505210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 493/1007] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:21:57 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:21:58.883048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a406-c029-4bfb-a400-31a579e1b15b None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:58.883547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a406-c029-4bfb-a400-31a579e1b15b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:58.883547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a406-c029-4bfb-a400-31a579e1b15b None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:58.884870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 515/1008] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:58.890429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:58.891168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:58.891591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:58.897189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:58.908718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:58.911117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:58.929590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815049a-f986-47dc-b116-01580da25aa2 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:58.929881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815049a-f986-47dc-b116-01580da25aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:58.930380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815049a-f986-47dc-b116-01580da25aa2 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:58.931754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 516/1009] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:21:58.936754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:58.937038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:58.937339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:58.942453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:58.954415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:58.955593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:58.980301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 494/1010] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:59.002248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:59.030315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 517/1011] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:59.053051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:59.120272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e3d9e-b834-45eb-af12-371cec54f996 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:59.153515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:59.158717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1824-ecda-4903-8671-025dabd6a202 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:59.162775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:59.162775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.162775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.162775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.163128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:59.163128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9fd6e-0b4f-4140-8e59-8076cf9a62b3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:59.189941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:59.195350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:59.196135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.196416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.196686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.197870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:59.199289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:59.200897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 518/1012] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:59.206627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:59.268830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 495/1013] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:59 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:21:59.307491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef168-940e-41ac-9cce-6b0468ac327c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:59.351347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:59.352023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.352302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.352571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.353331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:59.354790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3461d6-ade5-46ad-8f4c-bb8fe6ac41e7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:59.440843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 519/1014] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:59 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:59.538235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427db0f-f6e7-48b1-b022-068bc92bb956 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.538527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427db0f-f6e7-48b1-b022-068bc92bb956 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.539023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427db0f-f6e7-48b1-b022-068bc92bb956 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.540600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 496/1015] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:59.546019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.546320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.546602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.565244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:59.566472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:59.614841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:59.639111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 520/1016] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:59 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:59.661818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427db0f-f6e7-48b1-b022-068bc92bb956 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:59.765633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427db0f-f6e7-48b1-b022-068bc92bb956 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:59.801470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:59.802274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.802564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.803402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.804060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:59.805447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:59.817348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:59.818431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:21:59.819344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 497/1017] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:21:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:59.825195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:59.923346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae14fa2-f01c-45f6-834c-c055ef97c4dc None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:59.958331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:59.959034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:59.959323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:59.959602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:59.960310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:59.960622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:59.962018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:59.971514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 521/1018] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:59.972665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:00.029704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:00.076103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a45e5-46ea-425c-92cc-3f93302ea455 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:00.110353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:00.111019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:00.111320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:00.111588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:00.112296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:00.112601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:00.114038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962768c0-1466-4e1c-a5e0-8277e5a48236 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:00.123813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 498/1019] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Jul 12 14:21:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:00.125494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:00.171987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:00.227390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec841ae-262a-4c97-b07e-74ad87fcc8e6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:00.260655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:00.261637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:00.261928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:00.262201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:00.263014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:00.264375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf6ebf-da71-4837-a631-75bcf0497c3b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:00.344620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 522/1020] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:22:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:22:01.354958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b705a-9c39-47f6-b39d-29132920570c None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.355481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b705a-9c39-47f6-b39d-29132920570c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.355562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b705a-9c39-47f6-b39d-29132920570c None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.357026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 499/1021] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:01.362054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed70ff-c084-46fb-b761-e8d92d02e737 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.362483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed70ff-c084-46fb-b761-e8d92d02e737 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.362652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed70ff-c084-46fb-b761-e8d92d02e737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.379929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bbed70ff-c084-46fb-b761-e8d92d02e737 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:01.381186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bbed70ff-c084-46fb-b761-e8d92d02e737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:01.477241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 523/1022] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:01.499879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b705a-9c39-47f6-b39d-29132920570c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:01.514612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92c269-3432-4dd5-bfae-9dda72d916ce None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.514902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92c269-3432-4dd5-bfae-9dda72d916ce None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.515181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92c269-3432-4dd5-bfae-9dda72d916ce None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.516700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 524/1023] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:22:01.521844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.522138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.522422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.541578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:01.542772 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:01.563087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:01.611275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b705a-9c39-47f6-b39d-29132920570c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:01.613834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 525/1024] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:01.631971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:01.638586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:01.648273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:01.649067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.649381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.649662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] PATH_INFO: `/v3/services/image` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.650319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:01.651686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:01.663730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:01.664927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:01.665840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 500/1025] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Jul 12 14:22:01 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:01.671296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:01.750812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950655d9-4763-4ea0-97d9-cb944e5dcdab None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:01.784174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:01.784789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402562-2707-44c1-8d8d-af9801a4cceb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:01.784969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.785270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.785549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.786721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:01.788776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:01.790587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 526/1026] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:01.795922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:01.819637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:01.819821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:01.820296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.820581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.820861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.821596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] RBAC: Adding query filter params (name=image) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:01.821898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:01.823260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:01.828225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 501/1027] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:01.834497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:01.887322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:01.905079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e01568-46e3-4914-ba9e-9a0702b3b4a4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:01.940432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b411d2-f98d-4687-b6c8-7c7e22360d0b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:01.942246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:01.943925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.944434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.944908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.945854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:01.947888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:01.976483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:01.977148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:01.977463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:01.977741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:01.978447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:01.978757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:01.980168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:01.985026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 502/1028] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:01.990709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:02.049188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 527/1029] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:22:01 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:22:02.109537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33ac51-3b71-46c8-89cf-812541f164b2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:02.123867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:02.148269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:02.148860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.149221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.149641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.150464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] RBAC: Adding query filter params (interface=public, service_id=a8121bc15d8e4702994549c40a27e680, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:02.150775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:02.152166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b53999-5e96-46a4-b3f5-953900bc3729 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:02.162873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 503/1030] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:22:01 2022] GET /identity/v3/endpoints?service_id=a8121bc15d8e4702994549c40a27e680&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:02.163953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:02.250975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:02.274757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5c7e-be28-4cba-b065-6bf6e38a1cbe None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:02.309524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:02.310197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.310482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.310756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.311436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:02.312814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83be14-0152-4b33-821c-ce26229294ac None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:02.321042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 528/1031] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:22:02 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:22:02.467011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7a58-1984-4ebc-a916-5010be337f90 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.467557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7a58-1984-4ebc-a916-5010be337f90 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.467557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7a58-1984-4ebc-a916-5010be337f90 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.469031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 504/1032] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:02.474494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.474949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.475162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.493088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:02.494362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:02.563746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 529/1033] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:02.587480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b7a58-1984-4ebc-a916-5010be337f90 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:02.687420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b7a58-1984-4ebc-a916-5010be337f90 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:02.720096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:02.720855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.721132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.721425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.731770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:02.739716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:02.741265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:02.742144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:02.746335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 505/1034] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:22:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:02.747684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:02.819292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:02.847908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3206-3f25-4bea-8971-a80e903e2ad1 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:02.882765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:02.883433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:02.883717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:02.883991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:02.884735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:02.885041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:02.886685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:02.898908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 530/1035] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:22:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:02.900627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:02.937057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:03.000009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4026-ff93-4e89-bb62-a984b28df896 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:03.033241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:03.034034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:03.034312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:03.034582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:03.045433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:03.046866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:03.064674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:03.065736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:03.066672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:03.066672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:03.066672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:03.066672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:03.069490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 506/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:03.071723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:03.175295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ff9d-a22c-4129-8745-edb52a12cd35 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:03.207660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:03.208328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:03.208610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:03.209447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:03.210162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:03.210469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:03.211832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:03.213288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:03.224997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 531/1037] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:03.230690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:03.319623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:03.331795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06717-66d6-4da3-92fe-78ac4af11ebd None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:03.365866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:03.366654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:03.366937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:03.367213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:03.374783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:03.376214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:03.390392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:03.391448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:03.392415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:03.398019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:03.398612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 507/1038] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:22:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:03.512259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fc1f8-8ee1-4b78-9b19-024de045e132 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:03.546715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:03.547385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:03.547670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:03.547949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:03.548781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:03.549091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:03.550556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:03.572600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 532/1039] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:22:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:03.574483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:03.674284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756509e-2a28-4393-828a-6fa66e888121 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:03.707515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:03.708166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:03.708451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:03.709226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:03.730427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:03.780741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:03.782222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:03.880662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 508/1040] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:03.886559 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:03.942575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:03.989303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5801c33-674c-4556-844e-a80a9b639dc3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.023607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.024402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.024688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.024972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.035216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.036738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-49e3ee5f-08db-47ca-a96b-a6478bca0db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:04.054512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:04.055441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:04.061029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:04.061029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:04.061029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:04.061029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:04.061029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 533/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:04.062885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.112995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.150154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccf247-6e30-4a84-a8a8-21784c33cf79 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.150347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccf247-6e30-4a84-a8a8-21784c33cf79 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.150619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccf247-6e30-4a84-a8a8-21784c33cf79 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.152418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 534/1042] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:22:04.157018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.157348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.157627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.165577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12791-e678-4f67-b45d-989488f7d384 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.174496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:04.175665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:04.203756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.204894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.205416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.205896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.207253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:04.207813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.210930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.230012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 509/1043] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:04.231254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.257222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 535/1044] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:04.280489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.319582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.336399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2bba5-f683-481f-9eca-00f3c3b5bf0c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.346031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.372949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.373623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.373908 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.374198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.381004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.382477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.391853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e298a-4b58-4e3e-88e0-900ef37d1f19 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d4c1b148-c21e-4430-8992-522e3a7a2d15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:04.398397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:04.399496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 510/1045] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:04.400884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e594d4-bb19-48a4-ad9b-2de288c7e1ea None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.401499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e594d4-bb19-48a4-ad9b-2de288c7e1ea None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.401499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e594d4-bb19-48a4-ad9b-2de288c7e1ea None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.402712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 511/1046] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:04.405271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e594d4-bb19-48a4-ad9b-2de288c7e1ea None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.431186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.431975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.432256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.432530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.446279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f67677e4-1b5e-4cab-9735-7c7442c4517a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:04.453983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:04.460330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 536/1047] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:04.461470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.462461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.462461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.478595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:04.479770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:04.510375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e594d4-bb19-48a4-ad9b-2de288c7e1ea None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.544350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.545010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.545327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.545429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 537/1048] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:04.545912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.546224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.546680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:04.546992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.548422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.560791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 512/1049] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:04.566655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.571552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.595321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.644614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3846d-a6cf-4400-94b6-7a1a58b49283 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.667004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ea5a9-a744-441a-a017-6b4e85605eff None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.680765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.681453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.681736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.682010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.682750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:04.683058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.684405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.692229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 538/1050] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:04.693014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.699246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.699903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.700186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.700464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.701334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] RBAC: Adding query filter params (role.id=3e43f773e9c94aa5b6d013da39189b16, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:04.701649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.703067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.709195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 513/1051] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3e43f773e9c94aa5b6d013da39189b16 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:04.709777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.794615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.812111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3e60-f5ff-41af-ae61-ec4d24283cd7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.812328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:04.822555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0482b-f37a-4ec5-b5c5-7740f3c742a9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:04.850476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.851630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.851986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.852333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] PATH_INFO: `/v3/services/image` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.852996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.854510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.859331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:04.860456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:04.860738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:04.861207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] PATH_INFO: `/v3/users/cc79ff32cd2341dab983f070976c2449` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:04.867170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:04.868159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:04.870364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 539/1052] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:04.874444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.885313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] RBAC: Authorizing `identity:get_user(user_id=cc79ff32cd2341dab983f070976c2449)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:04.887025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:04.893241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 514/1053] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/users/cc79ff32cd2341dab983f070976c2449 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:04.894734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:04.981316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aec04-49f1-461d-b195-6cd415025c97 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.000153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586181-52ea-4105-b33c-9d7fb1f3a015 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.016477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.017118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.017428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.017702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.018403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] RBAC: Adding query filter params (name=image) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:05.018758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.020108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.030366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 540/1054] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:05.031585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:05.037629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.038280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.038565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.038848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.047703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:05.057590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:05.107069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.108509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.134885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d67ee-e193-4217-b81b-33d358c19ace None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.172164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.173182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.173691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.173995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.174701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:05.175125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.176502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.181744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 541/1055] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:05.187097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:05.214724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 515/1056] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:05.220300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:05.272842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:05.278100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:05.299404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e439d6-ad73-4d13-8491-7e48baa223c7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.324232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1f9f0-9b8a-4d65-9a4d-6363cd619c2d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.346670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.347438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.347720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.348003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.348727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.350130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec676f-9544-401e-ac38-bbe49b0a8d14 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.361315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.362178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.362464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.362739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.363916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.365401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.367110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 516/1057] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:05 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:05.372939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:05.416827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 542/1058] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:22:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:22:05.487651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0cc9e-790d-461f-ab6d-555204b2fa23 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:05.508207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:05.532235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:05.532820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:05.533093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:05.533404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:05.534266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=72fefd466ec74da988d7cdd974ec3622, user.id=cc79ff32cd2341dab983f070976c2449) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:05.535015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:05.536989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ae82f-d814-4dff-b03b-2234163d1fa6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:05.549189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 517/1059] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:05 2022] GET /identity/v3/role_assignments?user.id=cc79ff32cd2341dab983f070976c2449&scope.project.id=72fefd466ec74da988d7cdd974ec3622&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:06.721057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688ac7-4a79-438d-830f-ee2069fbd092 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:06.721655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688ac7-4a79-438d-830f-ee2069fbd092 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:06.721655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688ac7-4a79-438d-830f-ee2069fbd092 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:06.722963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 543/1060] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:06.728643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:06.729020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:06.729285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:06.746388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:06.747529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:06.816150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 518/1061] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:06.838241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68688ac7-4a79-438d-830f-ee2069fbd092 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:06.938785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68688ac7-4a79-438d-830f-ee2069fbd092 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:06.971421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:06.972149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:06.972427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:06.972699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:06.974592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:06.984110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:06.992796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:07.007538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:07.008455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 544/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:22:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:07.009223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.100856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c711345-10cd-427f-9621-e7bcf6ac2f05 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.110291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.135494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.136587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.136981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.137533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.138300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:07.138605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.139966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.152832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 519/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:22:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:07.154173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.256199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616b79b-101c-446f-bbba-cb9f22299dc5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.289663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.290449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.290728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.290995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.301379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.303377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:07.321167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:07.322123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:07.323068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:07.323068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:07.323068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:07.323068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:07.325502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 545/1064] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:07.328208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.431950 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc528-b762-47bd-8936-94fcac3cbc71 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.468633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.469333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.469622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.469897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.470622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:07.470933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.472299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dfc55-3850-48bd-8d18-ce4fe9293b24 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.490684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 520/1065] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:07.491842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.540816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.545539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0a88-bac1-4d84-ac92-b80613c1309f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.545846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0a88-bac1-4d84-ac92-b80613c1309f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.546132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf0a88-bac1-4d84-ac92-b80613c1309f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.547494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 521/1066] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:07.548557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181ae6-b0af-4197-a5f6-a1e3ed14dcdf None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.548843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181ae6-b0af-4197-a5f6-a1e3ed14dcdf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.549123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181ae6-b0af-4197-a5f6-a1e3ed14dcdf None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.550365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 522/1067] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:07.555162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b47fc-ef14-4fb3-9171-b0eb83b40be3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.555511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b47fc-ef14-4fb3-9171-b0eb83b40be3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.555794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b47fc-ef14-4fb3-9171-b0eb83b40be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.572102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-213b47fc-ef14-4fb3-9171-b0eb83b40be3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:07.573335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-213b47fc-ef14-4fb3-9171-b0eb83b40be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:07.597022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.600001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866f8b-1ddb-4cfa-aedf-4d8c7023518b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.632862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.633745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.634035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.634316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.641079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 523/1068] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:07.642351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.642499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.642689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.642916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.643900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:07.659729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:07.660726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:07.662332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 546/1069] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:07.664861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.667418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:07.668604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:07.734419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 524/1070] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:07.735238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.764533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b833b8-f62d-4d5b-a956-e345510f0439 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.771931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.798053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.798632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.798917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.799203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.826226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.843550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3f2e-71b4-4165-91ac-b8971b290a10 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:07.866431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.867867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.878352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:07.879028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:07.879314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:07.879595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:07.880437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:07.880754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:07.882213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:07.899961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 525/1071] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:07.900766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.977862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 547/1072] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:07.978536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:07.986329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:07.999217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b0af-32e5-4fd1-810d-a98faf7f4380 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.033043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.033923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.034209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.034485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.044517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:08.052025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:08.052981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:08.054144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 526/1073] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:08.055249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.069104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.077214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58a23c-991c-447a-8924-13aaebda3daa None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.110305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.111091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.111374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.111657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.121429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.122856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:08.140533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:08.141549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 548/1074] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:08.142547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.159021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad04f1-c39e-485d-b030-7c6a5e08559a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.194931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.195710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.195990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.196261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.202401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.206964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.208986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:08.225983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:08.227137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 527/1075] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:08.228070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.246378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a355b31-6723-4f30-9262-c7ad90ceb3da None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.287591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.289124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.289819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.290107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.290386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.291130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:08.291517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.292799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.301536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 549/1076] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:08.302770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.329535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dc2f-bacf-40de-abea-7b8b8802b9a3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.362698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.363368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.363657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.363952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.365495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:08.365814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.367175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.378735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 528/1077] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:08.379522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.408726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fa353-aff1-496c-bb32-fea2054f1a87 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.431800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.446847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.447867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.447867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.448075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.448776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:08.449087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.450490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.462502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 550/1078] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:08.463366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.526362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149d4d-b3c8-4e34-b8a9-4f87842355cb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.549408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.564975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.566302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.566302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.566524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] PATH_INFO: `/v3/users/cc79ff32cd2341dab983f070976c2449` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.567642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cc79ff32cd2341dab983f070976c2449)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.569286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.576418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 529/1079] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/users/cc79ff32cd2341dab983f070976c2449 => generated 285 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:08.577550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.608367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bbeb-bdd6-43bb-8f0c-6f89dbc939e3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.648394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.649225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.649499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.649767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.659673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.661171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:08.677615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:08.678563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:08.679503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 551/1080] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:08.684938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.696876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a069dd-5f84-4488-a95e-5b7983aaa6e3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.736834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.740573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.741478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.741756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.742028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.743213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.744639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.746438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 530/1081] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:08.752220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.795335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead9bc-e4cb-4080-8e2b-0a359ca2b483 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.810311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.828410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.828993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.829309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.829589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.830374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:08.830681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.832133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.844485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 552/1082] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:08.850410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:08.853425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a235-ea5a-4dba-b9a4-51a30342b6a4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.887032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.887914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.888204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.888478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622/users/cc79ff32cd2341dab983f070976c2449/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.890449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fefd466ec74da988d7cdd974ec3622, user_id=cc79ff32cd2341dab983f070976c2449, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.891867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c140a-673e-43d8-ae9d-0ad8437ae2e0 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:08.943354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:08.946185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634446fb-c6e8-4aa6-9d87-4f54a2f10be1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:08.955484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 531/1083] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:08 2022] PUT /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622/users/cc79ff32cd2341dab983f070976c2449/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:22:08.982135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:08.983215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:08.983502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:08.983781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/3e43f773e9c94aa5b6d013da39189b16` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:08.986131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=a6b4ed458d984a51829114a73534fce2, role_id=3e43f773e9c94aa5b6d013da39189b16)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:08.987600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4794134-8832-4ffd-90a9-affee2b89f7f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:09.053567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 553/1084] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:08 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/a6b4ed458d984a51829114a73534fce2/roles/3e43f773e9c94aa5b6d013da39189b16 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:22:10.946502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba3b-a46d-400f-a0ae-00672675ba24 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:10.947021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba3b-a46d-400f-a0ae-00672675ba24 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:10.947115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba3b-a46d-400f-a0ae-00672675ba24 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:10.948427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 532/1085] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:10.953803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:10.954187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:10.954366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:10.970393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:10.971581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:11.037116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 554/1086] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:11.058506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ed57-1b50-4596-a55a-5a01ff67e629 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.058868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ed57-1b50-4596-a55a-5a01ff67e629 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.059210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ed57-1b50-4596-a55a-5a01ff67e629 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.060687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.060998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 533/1087] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:11.065721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.066004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.066280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.082670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:11.083841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:11.149689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.151035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 534/1088] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:11.158958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00128ba-5221-49dd-91f4-caf74c226c66 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.173130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.191671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.193223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.194394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.194739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.205337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:11.213417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:11.214381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:11.215261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 555/1089] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:11.220069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.220558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.272653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad5c6f-e90a-4b91-9f31-ffaf5f0fb911 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.305876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.306677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.306957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.307230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.318200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:11.318331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9d08-653b-41ac-83b8-3003bd1a29a0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:11.326466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:11.327602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:11.328494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 535/1090] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:11.333959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.354364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.355021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.355304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.355577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.356314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:11.356615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.357994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.365804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 556/1091] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:11.371301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.386536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.436785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352da93-891d-42fc-b0a8-dd2ef6e5df0d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.444666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.471533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.472124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.472406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.472681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.473481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:11.473810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.475168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.475803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d201f-794a-4adc-a157-22b638a99520 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.483526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 536/1092] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:11.489991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.509918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.511047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.511327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.511606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] PATH_INFO: `/v3/users/cc79ff32cd2341dab983f070976c2449` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.512785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] RBAC: Authorizing `identity:get_user(user_id=cc79ff32cd2341dab983f070976c2449)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.514234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.515851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 557/1093] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/users/cc79ff32cd2341dab983f070976c2449 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:11.521390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.588970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccd0b-ba0c-4838-94b3-aeef35ebe0e2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.600376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.619120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad5072-379e-43d2-86ab-66e61c4549cc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.622236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.623021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.623299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.623574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.634091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.635762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.652578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.653246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:11.653358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:11.654213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:11.655131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:11.655131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:11.655131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:11.655131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:11.655369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.655369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.659196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 537/1094] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:11.659983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.682352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.723634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.725028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.760914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67126-dc7d-41d3-a188-bcc1b80be201 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.795012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.795664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.795949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.796225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.796944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:11.797274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.798625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.802867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.810562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 538/1095] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:11.815980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.821670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 558/1096] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:11.827478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.893697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:11.912537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaf0c2-7a42-4a9a-9c16-fdbbb539046e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.923992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62b232-cbe2-4bcc-9990-4bfd654870fc None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:11.946029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.946812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.947355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.947355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.954674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.956093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.957726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:11.958602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:11.958888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:11.959166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:11.960354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:11.961809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:11.963502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 559/1097] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:11.969450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:11.969843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:11.970964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:11.971949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 539/1098] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:11.976937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:12.021277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:12.071531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865981a1-1154-43e7-b2b0-c465fca8a72c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.076264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f86d-f319-4e32-892f-93511b7ef053 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.105030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.105705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.110944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.110944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.110944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=72fefd466ec74da988d7cdd974ec3622, user.id=cc79ff32cd2341dab983f070976c2449) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:12.110944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.110944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.111371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.111371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.111371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.111371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.111642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:12.111760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.113198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.115144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 560/1099] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/role_assignments?user.id=cc79ff32cd2341dab983f070976c2449&scope.project.id=72fefd466ec74da988d7cdd974ec3622&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:12.124269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:12.133917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 540/1100] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Jul 12 14:22:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:12.140150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:12.243962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775b46-1da6-40e6-882f-d49b5205f3bd None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.257793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:12.279488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.280132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.280410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.280679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.355630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.357073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.454400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 561/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:12.460653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:12.557299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac495ed-7326-48d4-a556-b8194cd5c11e None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.590434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.591217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.591495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.591764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.601678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.603111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:12.619946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:12.620841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:12.622197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:12.622197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:12.622197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:12.622197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:12.622197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 541/1102] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:12.626677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:12.641403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:12.724233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c30-af53-4221-a94e-d3a91eb740c7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.756655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.757329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.757612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.757887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.758601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:12.758905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.760255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.772067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 562/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:12.777633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:12.825792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:12.876646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c4eb5-f1d5-40d1-af6c-fbfe59d5aa89 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:12.913073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:12.913896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:12.914185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:12.914461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:12.920969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:12.922445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:12.936415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:12.937597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:12.943944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 542/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:12.944464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:13.012079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:13.040456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52041f7f-ebcc-4eff-af54-4571a4062c08 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:13.073332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:13.073989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:13.074274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:13.074551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:13.075338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:13.075648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:13.077140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35465590-d4a5-465a-b315-52980c0e6c8d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:13.088805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 563/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:13.094620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:13.195050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233765a-4b81-4ecd-8e07-d096cf85b670 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:13.205409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:13.227600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:13.228255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:13.228536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:13.228807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:13.229680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] RBAC: Adding query filter params (role.id=3e43f773e9c94aa5b6d013da39189b16, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=a6b4ed458d984a51829114a73534fce2) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:13.229992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:13.231515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6d569-6ca0-4c23-b3d6-97ff3ab1dae5 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:13.249260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 543/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:13 2022] GET /identity/v3/role_assignments?user.id=a6b4ed458d984a51829114a73534fce2&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3e43f773e9c94aa5b6d013da39189b16 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:14.163135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1a37-a636-418a-9a39-456de41da3c6 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:14.163695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1a37-a636-418a-9a39-456de41da3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:14.163695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1a37-a636-418a-9a39-456de41da3c6 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:14.165057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 564/1107] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:14.170189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:14.170606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:14.170740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:14.187059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:14.188229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:14.252519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 544/1108] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:14.277527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1a37-a636-418a-9a39-456de41da3c6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:14.388065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1a37-a636-418a-9a39-456de41da3c6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:14.415888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:14.420509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:14.421312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:14.421593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:14.421864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:14.426500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:14.426500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6d19-e32e-4e69-9411-a30b8ba9fe8b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:14.426500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 565/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:14.432335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:14.528184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507ae5-3ed7-4b86-ba5a-1e47e0f0262d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:14.530405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:14.561342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:14.562003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:14.562316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:14.562775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:14.563930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:14.565380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529847ff-d610-4542-a031-dedf850d9b10 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:14.657808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 545/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:22:14 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:16.685081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5828c-3f43-4b82-90dc-4ad941ce1239 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:16.685753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5828c-3f43-4b82-90dc-4ad941ce1239 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:16.685753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5828c-3f43-4b82-90dc-4ad941ce1239 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:16.687048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 566/1111] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:16.692148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:16.692565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:16.692676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:16.709411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:16.710572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:16.774125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 546/1112] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:16.797771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5828c-3f43-4b82-90dc-4ad941ce1239 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:16.896758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5828c-3f43-4b82-90dc-4ad941ce1239 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:16.929398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:16.930240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:16.930522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:16.930799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] PATH_INFO: `/v3/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:16.932291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] RBAC: Authorizing `identity:get_role(role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:16.933675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:16.935035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 567/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:16 2022] GET /identity/v3/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:16.940824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:17.027184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:17.036276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9f547-7759-4443-bf7d-bd734b50562d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:17.068665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:17.069550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:17.069833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:17.070105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] PATH_INFO: `/v3/users/b953dff1fd6d424388dc9283bbc8651b` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:17.082701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] RBAC: Authorizing `identity:get_user(user_id=b953dff1fd6d424388dc9283bbc8651b)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:17.084719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:17.086764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 547/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:16 2022] GET /identity/v3/users/b953dff1fd6d424388dc9283bbc8651b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:17.091996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:17.160058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:17.193996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c463e9b-2153-484b-a5b9-596efdf9a644 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:17.226321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:17.226951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:17.227236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:17.227514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:17.291215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:17.292581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:17.353403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:17.386193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 568/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:17.392061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:17.489781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b627-c43d-4e26-ab08-a6b748ba7911 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:17.522755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:17.523603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:17.523883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:17.524154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:17.525352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:17.526765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b689d-34f0-462b-8a69-c994d2558c7a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:17.528453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 548/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:17 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:17.534498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:17.631032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a1e7-3448-4082-a46d-41e747869226 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:17.664518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:17.665177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:17.665465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:17.665739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:17.666556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=72fefd466ec74da988d7cdd974ec3622, user.id=b953dff1fd6d424388dc9283bbc8651b) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:17.666861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:17.668271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd812f-7166-47eb-a111-c70a401ffe75 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:17.673497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 569/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:17 2022] GET /identity/v3/role_assignments?user.id=b953dff1fd6d424388dc9283bbc8651b&scope.project.id=72fefd466ec74da988d7cdd974ec3622&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:19.647463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b93-c382-4956-a2fa-7773c2f15989 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:19.648029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b93-c382-4956-a2fa-7773c2f15989 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:19.648029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68b93-c382-4956-a2fa-7773c2f15989 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:19.649407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 549/1118] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:19.654492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:19.654945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:19.655049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:19.672351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:19.673553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:19.697857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:19.737168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 570/1119] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:19.759426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68b93-c382-4956-a2fa-7773c2f15989 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:19.777608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:19.857920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68b93-c382-4956-a2fa-7773c2f15989 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:19.889637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:19.890468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:19.890755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:19.891028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] PATH_INFO: `/v3/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:19.892475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] RBAC: Authorizing `identity:get_role(role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:19.893854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:19.895729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 550/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:19 2022] GET /identity/v3/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:19.901521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:19.994517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabaa84-c656-4d78-b5f1-5512d134ce50 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:20.027208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:20.028043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:20.028323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:20.028593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] PATH_INFO: `/v3/users/b953dff1fd6d424388dc9283bbc8651b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:20.029770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] RBAC: Authorizing `identity:get_user(user_id=b953dff1fd6d424388dc9283bbc8651b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:20.031173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:20.032745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 571/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:19 2022] GET /identity/v3/users/b953dff1fd6d424388dc9283bbc8651b => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:20.038306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:20.120027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:20.135770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01e2d-52ae-46e7-a6f3-a75750e47013 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:20.168724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:20.169595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:20.169880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:20.170159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:20.171330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:20.172745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976fcb9-28d9-4af7-a164-9895f5dbe7aa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:20.174634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 551/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:20 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:20.180503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:20.210777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:20.277227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0108c-8efb-4153-afe0-5fee1af0c317 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:20.309903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:20.310778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:20.311064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:20.311338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622/users/b953dff1fd6d424388dc9283bbc8651b/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:20.313555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] RBAC: Authorizing `identity:create_grant(project_id=72fefd466ec74da988d7cdd974ec3622, user_id=b953dff1fd6d424388dc9283bbc8651b, role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:20.314983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506481-2728-4355-a0a4-c2805cef5283 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:20.374087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 572/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:20 2022] PUT /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622/users/b953dff1fd6d424388dc9283bbc8651b/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:22:22.406665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adeb87-5662-4e2e-9d7a-56e3988408c9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:22.407219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adeb87-5662-4e2e-9d7a-56e3988408c9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:22.407219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adeb87-5662-4e2e-9d7a-56e3988408c9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:22.408593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 552/1124] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:22.413759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:22.414151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:22.414322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:22.430244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:22.431427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:22.496124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 573/1125] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:22.520615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adeb87-5662-4e2e-9d7a-56e3988408c9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:22.617127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adeb87-5662-4e2e-9d7a-56e3988408c9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:22.650389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:22.651222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:22.651505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:22.651784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] PATH_INFO: `/v3/roles/7a8fbea92d6c454f8d93ec651319e770` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:22.653273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] RBAC: Authorizing `identity:get_role(role_id=7a8fbea92d6c454f8d93ec651319e770)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:22.654634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:22.655968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 553/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:22 2022] GET /identity/v3/roles/7a8fbea92d6c454f8d93ec651319e770 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:22.661560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:22.682702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:22.758921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7830-8130-4277-b619-d00dcea84ad3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:22.791572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:22.792422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:22.792705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:22.792975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] PATH_INFO: `/v3/users/b953dff1fd6d424388dc9283bbc8651b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:22.794184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] RBAC: Authorizing `identity:get_user(user_id=b953dff1fd6d424388dc9283bbc8651b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:22.795593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:22.797218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 574/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:22 2022] GET /identity/v3/users/b953dff1fd6d424388dc9283bbc8651b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:22.802854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:22.823768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:22.898875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b431f-cc1c-46da-93db-a938bcafafa9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:22.931373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:22.931951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:22.932234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:22.932513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:22.996311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:22.997773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:23.017670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:23.093720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 554/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:23.099676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:23.184235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:23.196060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec9a-8052-4c05-a057-f78aba3eb784 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:23.228938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:23.229808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:23.230089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:23.230357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] PATH_INFO: `/v3/projects/72fefd466ec74da988d7cdd974ec3622` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:23.231509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] RBAC: Authorizing `identity:get_project(project_id=72fefd466ec74da988d7cdd974ec3622)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:23.232916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c3ce-2af3-4be4-ae8c-1faa8d9d839f None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:23.234701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 575/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:23 2022] GET /identity/v3/projects/72fefd466ec74da988d7cdd974ec3622 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:23.240813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:23.343245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df743345-1f3a-4c26-bcf5-6b37da93086c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:23.362549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:23.376032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:23.376676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:23.376962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:23.377275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:23.378103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] RBAC: Adding query filter params (role.id=7a8fbea92d6c454f8d93ec651319e770, scope.project.id=72fefd466ec74da988d7cdd974ec3622, user.id=b953dff1fd6d424388dc9283bbc8651b) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:23.378414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:23.380359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5e3d3-29b5-4ac0-9556-92ba440eafa8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:23.386342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 555/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:23 2022] GET /identity/v3/role_assignments?user.id=b953dff1fd6d424388dc9283bbc8651b&scope.project.id=72fefd466ec74da988d7cdd974ec3622&role.id=7a8fbea92d6c454f8d93ec651319e770 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:25.412558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bb6e-221a-4d97-9bb8-0047fe6a46b4 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:25.413087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bb6e-221a-4d97-9bb8-0047fe6a46b4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:25.413087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2bb6e-221a-4d97-9bb8-0047fe6a46b4 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:25.422944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 576/1131] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:25.423039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:25.423039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:25.423039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:25.437250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:25.438432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:25.504468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 556/1132] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:25.527975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2bb6e-221a-4d97-9bb8-0047fe6a46b4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:25.629744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2bb6e-221a-4d97-9bb8-0047fe6a46b4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:25.662690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:25.663414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:25.663698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:25.663970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:25.665146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:25.666607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089b65-95e5-4f49-8eb7-41e02c1f10e5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:25.668203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 577/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:25.673972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:25.768791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4c9a-dd38-4cdb-85fc-264c63fe37d9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:25.782874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:25.802406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:25.803052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:25.803337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:25.803615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:25.804312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:25.805768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248bf7c-d5d3-4312-acd4-6393be2b96a8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:25.896027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 557/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:22:25 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:27.916806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d7d0-5b14-4999-b873-f1f06674fea5 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:27.917333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d7d0-5b14-4999-b873-f1f06674fea5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:27.917333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158d7d0-5b14-4999-b873-f1f06674fea5 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:27.918673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 578/1135] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:27.923831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:27.924204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:27.924392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:27.940776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:27.941988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:28.006503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 558/1136] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:28.030799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d7d0-5b14-4999-b873-f1f06674fea5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:28.056644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:28.130692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158d7d0-5b14-4999-b873-f1f06674fea5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:28.162262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:28.163036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:28.163320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:28.163596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:28.164762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:28.166216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e682-e717-49dc-bf8a-91cbee0cc156 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:28.167868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 579/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:28.173583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:28.263809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:28.269462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad1385-6fda-4199-9441-ff4ea063e701 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:28.303593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:28.304252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:28.304535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:28.304810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:28.305553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:28.307017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb4efd-b233-4208-b4e0-b2d73384e75d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:28.388745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 559/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:22:28 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:22:30.442578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aec43-f7b0-4f77-9dd9-2fe8fea05893 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:30.443023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aec43-f7b0-4f77-9dd9-2fe8fea05893 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:30.443127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aec43-f7b0-4f77-9dd9-2fe8fea05893 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:30.444492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 580/1139] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:30.449714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:30.450130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:30.450275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:30.466642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:30.467817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:30.534221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 560/1140] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:30.558369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aec43-f7b0-4f77-9dd9-2fe8fea05893 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:30.646398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:30.655477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aec43-f7b0-4f77-9dd9-2fe8fea05893 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:30.689036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:30.689836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:30.690119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:30.690394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:30.700159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:30.707634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:30.708663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:30.709765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 581/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:30.714722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:30.792401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:30.810318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262882fd-ded4-4e5e-8263-68cf6a7a30c3 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:30.843263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:30.843926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:30.844543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:30.844822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:30.845598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:30.845905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:30.847256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:30.855555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 561/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:30.861225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:30.958534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d2a9-7fe3-468d-946a-6b8cfdef8a19 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:30.990825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:30.991685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:30.991967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:30.992244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] PATH_INFO: `/v3/users/1a833a550ca347f89c756fdd6b89509b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:31.005012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] RBAC: Authorizing `identity:get_user(user_id=1a833a550ca347f89c756fdd6b89509b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:31.006469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:31.008083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 582/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:30 2022] GET /identity/v3/users/1a833a550ca347f89c756fdd6b89509b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:31.014253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:31.111089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fd1dc-3a8a-45af-a017-782e16e26743 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:31.142834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:31.143466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:31.143751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:31.144028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:31.158375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:31.209930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:31.211295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:31.305654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 562/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:31.311521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:31.323889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:31.407625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f34393-5c35-439f-aec0-115d5bce8f5e None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:31.439557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:31.440415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:31.440700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:31.440978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] PATH_INFO: `/v3/projects/f24e4a70c5a94a25b54dc625b681290c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:31.442713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] RBAC: Authorizing `identity:get_project(project_id=f24e4a70c5a94a25b54dc625b681290c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:31.444124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9812-74f6-49cd-b7b7-491f1fabbb5e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:31.445844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 583/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:31 2022] GET /identity/v3/projects/f24e4a70c5a94a25b54dc625b681290c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:31.451769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:31.496144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:31.547929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3f3ec-c4b2-4436-b0f2-eb337c5a1d1d None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:31.580318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:31.580948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:31.581259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:31.581539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:31.582966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=f24e4a70c5a94a25b54dc625b681290c, user.id=1a833a550ca347f89c756fdd6b89509b) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:31.583273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:31.584680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0d708-0409-4fdc-ad73-a3710806e456 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:31.589951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 563/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:31 2022] GET /identity/v3/role_assignments?user.id=1a833a550ca347f89c756fdd6b89509b&scope.project.id=f24e4a70c5a94a25b54dc625b681290c&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:33.563223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff9a-d7ba-4bce-8979-4a130d56550a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.563761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff9a-d7ba-4bce-8979-4a130d56550a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.563761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff9a-d7ba-4bce-8979-4a130d56550a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.565134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 584/1147] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:33.570753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.571196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.571292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.588324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:33.589544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:33.653337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 564/1148] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:33.674757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ff9a-d7ba-4bce-8979-4a130d56550a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.750286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:33.772194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ff9a-d7ba-4bce-8979-4a130d56550a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.803850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.804620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.804898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.805190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.814832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:33.822829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:33.823966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:33.825012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 585/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:33.830185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.869314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:33.928413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4137d-b91b-418a-ac30-e33e0ea2c4e4 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.960943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.961634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.961919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.962192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.962932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:33.963238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:33.964588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:33.972387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 565/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:33.978229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.073796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd33ee-329f-4212-9e19-91085464a195 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.104885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.105760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.106039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.106308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] PATH_INFO: `/v3/users/1a833a550ca347f89c756fdd6b89509b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.107461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] RBAC: Authorizing `identity:get_user(user_id=1a833a550ca347f89c756fdd6b89509b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.108867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.110481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 586/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/users/1a833a550ca347f89c756fdd6b89509b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:34.116854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.212815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.215877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26d30f-4718-4d6f-b172-bb34bdb3b032 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.248329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.249199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.249484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.249755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] PATH_INFO: `/v3/projects/f24e4a70c5a94a25b54dc625b681290c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.250925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] RBAC: Authorizing `identity:get_project(project_id=f24e4a70c5a94a25b54dc625b681290c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.252338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479be77a-1358-4f73-b8ef-8553caa56416 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.255059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 566/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:34 2022] GET /identity/v3/projects/f24e4a70c5a94a25b54dc625b681290c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:34.260855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.270189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.357365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d91e8-0112-4c5d-80a0-a6c71343d2aa None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.389715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.390591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.390874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.391157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] PATH_INFO: `/v3/projects/f24e4a70c5a94a25b54dc625b681290c/users/1a833a550ca347f89c756fdd6b89509b/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.393093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24e4a70c5a94a25b54dc625b681290c, user_id=1a833a550ca347f89c756fdd6b89509b, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.394540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f791-629e-4046-8870-37019fe2a849 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.474771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 587/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:34 2022] PUT /identity/v3/projects/f24e4a70c5a94a25b54dc625b681290c/users/1a833a550ca347f89c756fdd6b89509b/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:22:36.454810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f881-f1bd-47f1-8520-6b1761e5abb9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.455286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f881-f1bd-47f1-8520-6b1761e5abb9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.455381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8f881-f1bd-47f1-8520-6b1761e5abb9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.457278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 567/1154] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:36.462499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.462901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.463063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.480861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:36.482089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:36.539929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:36.547117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 588/1155] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:36.570898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8f881-f1bd-47f1-8520-6b1761e5abb9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:36.673213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8f881-f1bd-47f1-8520-6b1761e5abb9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:36.706955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:36.707663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.707946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.708223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.718573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:36.726664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:36.727829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:36.728736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 568/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:36.733956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:36.832660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95523b-730d-4288-8e1e-53408ff5b872 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:36.866038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:36.866695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.866979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.867257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.868000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:36.868307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:36.869726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:36.877662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 589/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:36.884273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:36.921462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:36.995662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebffc4-76f8-4380-99c7-f1d4a5e720da None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.029490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.030350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.030633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.030911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] PATH_INFO: `/v3/users/1a833a550ca347f89c756fdd6b89509b` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.032096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] RBAC: Authorizing `identity:get_user(user_id=1a833a550ca347f89c756fdd6b89509b)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.033556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.035204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 569/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:36 2022] GET /identity/v3/users/1a833a550ca347f89c756fdd6b89509b => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:37.040972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.098737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.138680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98afbae-821e-4505-9759-1d9c52128ee0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.171382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.172011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.172299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.172577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.236944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.238416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.294556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.336423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 590/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:37.342282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.433387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.439186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20093ad2-8876-4004-8b90-961cbd9900cb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.473585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.474442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.474730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.475011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] PATH_INFO: `/v3/projects/f24e4a70c5a94a25b54dc625b681290c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.476189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] RBAC: Authorizing `identity:get_project(project_id=f24e4a70c5a94a25b54dc625b681290c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.477652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2f74c-161d-4f7f-8eca-5c000a9a74db None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.479364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 570/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/projects/f24e4a70c5a94a25b54dc625b681290c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:37.485417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.580837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26958987-0b6a-4bc9-9999-3d009db2c942 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.614345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.614991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.615272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.615542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.616362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=f24e4a70c5a94a25b54dc625b681290c, user.id=1a833a550ca347f89c756fdd6b89509b) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:37.616669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.618173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe4d10-91ef-44e6-b65a-50120ec5f84e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.623743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 591/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/role_assignments?user.id=1a833a550ca347f89c756fdd6b89509b&scope.project.id=f24e4a70c5a94a25b54dc625b681290c&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:39.658802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e36ae-48cc-4680-9cd6-ad7b6180565a None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:39.659829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e36ae-48cc-4680-9cd6-ad7b6180565a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:39.659914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e36ae-48cc-4680-9cd6-ad7b6180565a None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:39.661278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 571/1162] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:39.666345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:39.666768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:39.666880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:39.683712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:39.684873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:39.700755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:39.748873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 592/1163] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:39.770946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e36ae-48cc-4680-9cd6-ad7b6180565a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:39.868369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e36ae-48cc-4680-9cd6-ad7b6180565a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:39.901595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:39.902219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:39.902499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:39.902921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:39.931787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:39.967177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:39.968538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:40.062304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 572/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:40.068233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:40.162146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e04a85-c545-4daa-b11b-5c162bc426a5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:40.193559 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:40.194334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:40.194618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:40.194887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:40.201299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:40.202713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-55e308d5-3eb1-4ca5-9f69-56f1774db836 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:40.216746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:40.217923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:40.218957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 593/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:22:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:40.224351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:40.284013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:40.320525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9a648-d5ae-41a8-9be0-783d1a8f25b2 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:40.353504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:40.354153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:40.354436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:40.354712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:40.355427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:40.355733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:40.357082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff5ea6-cb7f-4ec8-b8ad-4ef20e30864d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:40.364038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 573/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Jul 12 14:22:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:42.354960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fcdb0-e267-4a6c-877b-c38005f8299f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.355490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fcdb0-e267-4a6c-877b-c38005f8299f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.355490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fcdb0-e267-4a6c-877b-c38005f8299f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.356877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 594/1167] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:42.361955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803db28-a2a7-4c49-b15e-9383b306c26e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.362337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803db28-a2a7-4c49-b15e-9383b306c26e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.362548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803db28-a2a7-4c49-b15e-9383b306c26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.378756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2803db28-a2a7-4c49-b15e-9383b306c26e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:42.379929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2803db28-a2a7-4c49-b15e-9383b306c26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:42.443470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 574/1168] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:42.466151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fcdb0-e267-4a6c-877b-c38005f8299f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.481793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:42.565178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fcdb0-e267-4a6c-877b-c38005f8299f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.598361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.599005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.599283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.599555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.600282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.601666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ba4f0-fb13-4aee-9824-6475a15483d0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.668411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 595/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Tue Jul 12 14:22:42 2022] POST /identity/v3/domains => generated 267 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:44.690272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d07b3-19c6-4d67-ba29-0463610095d1 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:44.690793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d07b3-19c6-4d67-ba29-0463610095d1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:44.690914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d07b3-19c6-4d67-ba29-0463610095d1 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:44.692259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 575/1170] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:44.697353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:44.697733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:44.697909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:44.714602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:44.715764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:44.779394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 596/1171] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:44.801695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d07b3-19c6-4d67-ba29-0463610095d1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:44.859638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:44.901584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d07b3-19c6-4d67-ba29-0463610095d1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:44.933940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:44.934782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:44.935066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:44.935345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] PATH_INFO: `/v3/domains/4ef1392a21c64b71b11a360828173075` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:44.946133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ef1392a21c64b71b11a360828173075)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:44.947514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478c5a6-0265-4e2e-ad4c-e92c3a026065 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:44.949120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 576/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Jul 12 14:22:44 2022] GET /identity/v3/domains/4ef1392a21c64b71b11a360828173075 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:44.954817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:44.957598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:45.048975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de13803-dcf3-40a6-b1b5-dd4414aa1ec9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:45.081764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:45.082546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:45.083056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:45.083540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:45.084240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:45.085690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304acaf-3651-40ed-87b2-6b6fc8f4eb77 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:45.159866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 597/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:22:44 2022] POST /identity/v3/projects => generated 361 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:47.197110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcca76-f72c-4492-9d72-691170736487 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.197697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcca76-f72c-4492-9d72-691170736487 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.197806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcca76-f72c-4492-9d72-691170736487 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.199091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 577/1174] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:47.204121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.204491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.204676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.221061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:47.222789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:47.271103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.287449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 598/1175] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:47.311926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcca76-f72c-4492-9d72-691170736487 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.410376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcca76-f72c-4492-9d72-691170736487 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.426826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.443632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.444487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.444772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.445051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] PATH_INFO: `/v3/domains/4ef1392a21c64b71b11a360828173075` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.446253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ef1392a21c64b71b11a360828173075)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.447762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e83d-80e5-4732-9f48-9b170d3c3d23 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.449371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 578/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/domains/4ef1392a21c64b71b11a360828173075 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:47.455015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.550965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90692c65-f5e0-4df5-a0c9-e2a857961300 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.584272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.584924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.585238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.585521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.586225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.587641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771a366-9bc9-4345-b6da-8cfb6dd54982 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.661987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 599/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:22:47 2022] POST /identity/v3/users => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:49.716920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54b2cb-1a5b-48e1-bc2a-52fecfd726c2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:49.717501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54b2cb-1a5b-48e1-bc2a-52fecfd726c2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:49.717501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54b2cb-1a5b-48e1-bc2a-52fecfd726c2 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:49.718791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 579/1178] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:49.723886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:49.724334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:49.724431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:49.741648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:49.743112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:49.777034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:49.807661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 600/1179] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:49.831201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54b2cb-1a5b-48e1-bc2a-52fecfd726c2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:49.928915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54b2cb-1a5b-48e1-bc2a-52fecfd726c2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:49.960524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:49.961319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:49.961601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:49.961874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:49.972005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:49.979300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:49.980473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:49.981483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 580/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:49.986344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.088807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a48dd-d2ca-4970-9498-931e75312f6a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.121482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.122136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.122415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.122686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.123423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:50.123728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.125074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.132812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 601/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:50.138912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.184339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.235280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263121e7-5d22-4b8e-aa58-4995d8e85201 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.268039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.268880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.269189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.269467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] PATH_INFO: `/v3/users/488685c38a40422e9652a4829fc937a9` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.281932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] RBAC: Authorizing `identity:get_user(user_id=488685c38a40422e9652a4829fc937a9)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.283361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.285532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 581/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/users/488685c38a40422e9652a4829fc937a9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:50.291133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.321763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.387225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235de8-ab27-45d8-9681-24e094a5d312 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.419630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.420261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.420542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.420817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.563805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.565184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.593272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.659665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 602/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:50.665558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.762195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674445c0-8c26-4dbd-ab43-a14a54d0d165 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.764466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.795053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.795889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.796170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.796444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] PATH_INFO: `/v3/projects/93919fb12f1649a8a995f51561316f2a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.797623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] RBAC: Authorizing `identity:get_project(project_id=93919fb12f1649a8a995f51561316f2a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.799037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6398-fe80-4aa2-b035-bd26845c0098 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.800685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 582/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/projects/93919fb12f1649a8a995f51561316f2a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:50.806636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.900998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaaf62-d7ac-4d0a-8997-1da8b52f13e1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.928422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.933123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.933783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.934062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.934332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.935147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=93919fb12f1649a8a995f51561316f2a, user.id=488685c38a40422e9652a4829fc937a9) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:50.935456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.936853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3a39e-4f05-4eba-bf69-6e94124d2fbd None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.942579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 603/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/role_assignments?user.id=488685c38a40422e9652a4829fc937a9&scope.project.id=93919fb12f1649a8a995f51561316f2a&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:52.950586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f747-644a-485c-9284-c511435a1792 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:52.951117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f747-644a-485c-9284-c511435a1792 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:52.951117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f747-644a-485c-9284-c511435a1792 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:52.952496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 583/1186] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:52.958311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:52.958732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:52.958872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:52.976380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:52.977595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:53.041627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 604/1187] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:53.063337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07f747-644a-485c-9284-c511435a1792 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:53.164232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07f747-644a-485c-9284-c511435a1792 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:53.196537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:53.197343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:53.197625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:53.197899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:53.208230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:53.215632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:53.216739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:22:53.217841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 584/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:53.223150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:53.316293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb5564-e2ca-42e0-bd65-4c1827e768b8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:53.340565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:53.348685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:53.349362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:53.357133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:53.357133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:53.357133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:53.357133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:53.357133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:53.359877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 605/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:53.365537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:53.397235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:53.465335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d409c-61bf-4df5-a86b-ab45b152b976 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:53.497868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:53.498719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:53.499004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:53.499276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] PATH_INFO: `/v3/users/488685c38a40422e9652a4829fc937a9` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:53.500440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] RBAC: Authorizing `identity:get_user(user_id=488685c38a40422e9652a4829fc937a9)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:53.501884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:53.504048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 585/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:53 2022] GET /identity/v3/users/488685c38a40422e9652a4829fc937a9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:53.510293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:53.605499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2aabd-da33-4f14-ba21-f51d69b11a91 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:53.638549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:53.639390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:53.639675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:53.639960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] PATH_INFO: `/v3/projects/93919fb12f1649a8a995f51561316f2a` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:53.641132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] RBAC: Authorizing `identity:get_project(project_id=93919fb12f1649a8a995f51561316f2a)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:53.642579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e26b6-270b-4b17-9b3b-06cb8cd2de5e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:53.644244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 606/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:53 2022] GET /identity/v3/projects/93919fb12f1649a8a995f51561316f2a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:53.650875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:53.745004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:53.747786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea1f6-4f31-444b-8591-b3ecc4cd06f4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:53.780201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:53.781065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:53.781375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:53.781656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] PATH_INFO: `/v3/projects/93919fb12f1649a8a995f51561316f2a/users/488685c38a40422e9652a4829fc937a9/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:53.784113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] RBAC: Authorizing `identity:create_grant(project_id=93919fb12f1649a8a995f51561316f2a, user_id=488685c38a40422e9652a4829fc937a9, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:53.785562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bb4f8-e8f9-4b8a-8f4d-db979f4353eb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:53.869370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 586/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:53 2022] PUT /identity/v3/projects/93919fb12f1649a8a995f51561316f2a/users/488685c38a40422e9652a4829fc937a9/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:22:55.843223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746b19-f519-4a03-97a0-c2153c9ea099 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.843723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746b19-f519-4a03-97a0-c2153c9ea099 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.843723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746b19-f519-4a03-97a0-c2153c9ea099 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.845055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 607/1193] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:55.850881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.851247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.851440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.867638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:55.868815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:55.933017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 587/1194] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:55.956002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746b19-f519-4a03-97a0-c2153c9ea099 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.011728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.053996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746b19-f519-4a03-97a0-c2153c9ea099 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.084566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.085361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.085646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.085921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.096661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:56.104079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:56.105245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:22:56.106313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 608/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:56.111758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.196797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.205650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9600361-7a2a-4282-914a-2ab69e34b2f7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.239234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.239884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.240167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.240443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.241204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:56.241625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.242987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.251078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 588/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:56.256712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.352763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b005-d671-4380-b25a-b620459e1251 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.368252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.384967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.387134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.387134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.387134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] PATH_INFO: `/v3/users/488685c38a40422e9652a4829fc937a9` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.387522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] RBAC: Authorizing `identity:get_user(user_id=488685c38a40422e9652a4829fc937a9)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.388930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.390550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 609/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/users/488685c38a40422e9652a4829fc937a9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:56.396767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.494383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc80fcb-6b3c-4e7f-85e1-d187e9f5ee63 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.527000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.527640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.527925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.528202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.549204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.593697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.595132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.689273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 589/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:56.695135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.800201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115701-2a4c-41d1-8887-5527fca7f613 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.833245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.834094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.834377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.834652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] PATH_INFO: `/v3/projects/93919fb12f1649a8a995f51561316f2a` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.835828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=93919fb12f1649a8a995f51561316f2a)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.837294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd4bc8-3298-4207-aca8-9bbbdd14a6f6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.838994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 610/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/projects/93919fb12f1649a8a995f51561316f2a => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:56.845039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.896161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.941126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438bc5e-d74c-4cea-acf1-0cb8a07aabdb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.974149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.974780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.975062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.975339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.976162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=93919fb12f1649a8a995f51561316f2a, user.id=488685c38a40422e9652a4829fc937a9) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:56.976475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.977912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef966301-b15c-45c7-925f-64f35e852883 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.983516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 590/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/role_assignments?user.id=488685c38a40422e9652a4829fc937a9&scope.project.id=93919fb12f1649a8a995f51561316f2a&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:23:06.828546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=74 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:06.980960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=42 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:08.698773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edb7c-9a36-4de3-9284-ba23f6d8b78e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:08.699345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edb7c-9a36-4de3-9284-ba23f6d8b78e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:08.699345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499edb7c-9a36-4de3-9284-ba23f6d8b78e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:08.700640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 611/1201] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:08.705789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:08.706247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:08.706342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:08.725357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:08.726547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:08.791083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 591/1202] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:08.814673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499edb7c-9a36-4de3-9284-ba23f6d8b78e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:08.915678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499edb7c-9a36-4de3-9284-ba23f6d8b78e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:08.947860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:08.948635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:08.948965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:08.949219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:08.959988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:08.961457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:08.977864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:08.978884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:08.979899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:08.979899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:08.979899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:08.979899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:08.981479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 612/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:08.984978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:09.080862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a755a-478a-4800-ad5a-a811e4cfae88 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:09.107813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:09.113985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:09.114631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:09.114920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:09.115178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:09.115895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:09.116199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:09.117584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0adeb-bf82-4a31-b46a-1c32965aa4f7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:09.128794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 592/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:09.135199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:09.186856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:09.233291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acadf16-9be0-4b6b-b96e-4a09c7d627f7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:09.267172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:09.267838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:09.268126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:09.268404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:09.269121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:09.270726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f986c-355b-42e7-97cd-5e3563c24019 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:09.345534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 613/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Jul 12 14:24:09 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:24:11.399292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed4ae-fb3e-455d-8479-c0353091ec4a None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:11.399859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed4ae-fb3e-455d-8479-c0353091ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:11.399859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ed4ae-fb3e-455d-8479-c0353091ec4a None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:11.401275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 593/1206] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:11.406413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:11.406844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:11.406974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:11.423427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:11.424605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:11.489442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 614/1207] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:11.512814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ed4ae-fb3e-455d-8479-c0353091ec4a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:11.614752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ed4ae-fb3e-455d-8479-c0353091ec4a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:11.647464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:11.648251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:11.648534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:11.648812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:11.659184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:11.667178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:11.668354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:11.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 594/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:11.674365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:11.713673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:11.770857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86002e4-564f-443b-9be0-2a768c55ee38 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:11.803840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:11.804498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:11.804780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:11.805052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:11.805819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:11.806125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:11.807486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:11.815091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 615/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:11.821421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:11.871461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:11.919509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f3c3b-cca1-4792-84e8-d97972bb9a5a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:11.953332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:11.954119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:11.954409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:11.954688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:11.965044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:11.966526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:11.984204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:11.985326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:11.986470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:11.986470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:11.986470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:11.986470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:11.986470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 595/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:11.991166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.086271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:12.088944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642065c1-a43b-4680-88be-ed93c9615668 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:12.121436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:12.122090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.122374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.122649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.123366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:12.123674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:12.125065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:12.137838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 616/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:12.143459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.241293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975a6b1-c2c0-4943-af66-2338a25c01eb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:12.242811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:12.275484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:12.276266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.276549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.276827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.283947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:12.285762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:12.299693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:12.301407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:12.302503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 596/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:12.307372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.404523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd467199-607e-430e-9bbf-b48e1b0d631a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:12.438144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:12.438817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.439099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.439372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.440209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:12.440516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:12.446513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:12.458742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 617/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:12.464403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.563041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4e5b7-18f1-416d-9d3e-d1a9840aa061 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:12.594913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:12.595563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.595888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.596164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.625839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:12.662518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:12.663974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:12.758865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 597/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:24:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:24:12.764628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.783417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:12.861565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9aefa-932c-4e7d-8b20-c112afc44df1 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:12.894823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:12.895600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.895884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.896157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.905837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:12.907266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:12.924321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:12.926011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:12.927205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:12.927205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:12.927205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:12.927205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:12.927205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 618/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:12.931721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:12.992222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:13.030928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2ba54-b560-4bf5-9a56-016f0d913f62 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:13.064747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:13.065424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:13.065712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:13.066588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:13.067312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:13.067624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:13.068982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:13.080091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 598/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:13.085778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:13.154313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:13.183964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8b0d-869e-48c0-a5b1-f19001080615 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:13.215808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:13.216585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:13.218511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:13.218511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:13.224148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:13.225610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:13.239912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:13.240847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:13.247700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 619/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:13.248327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:13.345329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19eb0-596e-4b91-aa7e-d0ab4dd7d7e6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:13.366422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:13.377462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:13.378114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:13.379012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:13.379289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:13.380079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:13.380389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:13.382096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3bcb1-447c-4ca5-a60a-fda48412972d None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:13.393534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 599/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:13.400135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:13.508328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7a4e7-a70a-4ff4-89fe-61e62ef1f5a7 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:13.535339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:13.541080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:13.541756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:13.542037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:13.542312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:13.543133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=e40592fc25234d65a0be99d3d29869cb) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:13.543444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:13.544901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846c60-33da-4894-8536-a2dd658fa93d None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:13.550787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 620/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:24:13 2022] GET /identity/v3/role_assignments?user.id=e40592fc25234d65a0be99d3d29869cb&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:15.543583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c31ed-fcd8-435c-a826-b4518ddd566e None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.544184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c31ed-fcd8-435c-a826-b4518ddd566e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.544184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c31ed-fcd8-435c-a826-b4518ddd566e None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.545521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 600/1220] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:15.551247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.551679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.551915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.568324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:15.569614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:15.634949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 621/1221] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:15.656999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c31ed-fcd8-435c-a826-b4518ddd566e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:15.756458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c31ed-fcd8-435c-a826-b4518ddd566e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:15.790743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:15.791519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.791795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.792118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.802406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:15.809965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:15.811018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:15.811960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 601/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:15.817528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:15.914128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adeb41-2094-4a2f-85a1-56fa914fa649 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:15.938470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:15.948802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:15.949499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.949850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.950118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.950866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] RBAC: Adding query filter params (name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:15.951174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:15.952534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:15.960781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 622/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:24:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:15.966468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.009885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.062713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bdf20-0c2a-4b95-9186-d7127d210d6f None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.095188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.096312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.096312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.096486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.107077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.108494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:16.125449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:16.126524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:16.127676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:16.127676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:16.127676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:16.127676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:16.127676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 602/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:16.132725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.229297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4d28a-ed4e-4723-b38e-41c57e843073 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.262617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.263209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.263486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.263759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.264471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:16.264775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.266165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.278031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 623/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:24:16.284021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.375458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.382782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ffa5a-a760-4348-949f-2d39eb01763b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.415838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.416628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.416909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.417204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.424560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.426083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:16.440211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:16.441849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:16.443614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 603/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:16.447962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.494778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.545041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4843ca-ecfe-4491-b90a-7eba5814bae4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.579244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.579908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.580190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.580464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.581353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:16.581658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.583083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.599602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 624/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:16.605339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.703737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704b48-b9cb-4290-9f98-37c3677613ee None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.735880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.736657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.736942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.737239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.748037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.749497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:16.766972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:16.773568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:16.774721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:16.774721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:16.774721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:16.774721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:16.774721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 604/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:16.775029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.871999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4205b1d-e7ad-4c36-bcf9-8a80d1994df6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.874197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.904924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.905609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.905898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.906176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.906897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:16.907264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.908566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.920104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 625/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:16.926344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.935576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:17.023488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ab3b0-e20e-47e5-88aa-2d732ad2a848 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:17.055100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:17.055876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:17.056162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:17.056446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:17.062983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:17.064410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:17.079809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:17.083629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:17.084830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 605/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:17.087370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:17.183204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e2b6b-52f3-480a-9337-bf688064d9f5 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:17.216466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:17.220939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9585b-48bc-46ef-8c74-f04733162556 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:17.233214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 626/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:17.239781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:17.305932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:17.339813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e39b4-6899-4816-9117-9f264c733521 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:17.373649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:17.374528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:17.374816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:17.375096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/e40592fc25234d65a0be99d3d29869cb/roles/3fbeb7e9911f44ceaeffd3742d9c728c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:17.389833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=e40592fc25234d65a0be99d3d29869cb, role_id=3fbeb7e9911f44ceaeffd3742d9c728c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:17.391281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d81524-9ebd-4080-b39d-46d44c8af0c7 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:17.469305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 606/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:24:17 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/e40592fc25234d65a0be99d3d29869cb/roles/3fbeb7e9911f44ceaeffd3742d9c728c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:24:19.489610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e4a6-c010-45af-9040-91ce57f3a39d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.490124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e4a6-c010-45af-9040-91ce57f3a39d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.490218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09e4a6-c010-45af-9040-91ce57f3a39d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.491547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 627/1233] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:19.496687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.497114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.497288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.514046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:19.515219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:19.580563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 607/1234] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:19.604382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e4a6-c010-45af-9040-91ce57f3a39d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:19.630135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:19.702616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09e4a6-c010-45af-9040-91ce57f3a39d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:19.735824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:19.736609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.736891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.737184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.747794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:19.756351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:19.757540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:19.758798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 628/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Jul 12 14:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:19.763541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:19.808601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:19.860956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3803e-4245-4af6-8bb7-cc993ddaf286 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:19.894229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:19.894896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.895178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.895453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.896197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:19.896504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:19.897888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:19.906322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 608/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Jul 12 14:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:19.912506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:19.995649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:20.011004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c714591-4d52-4a09-bfa2-26eaa06efb37 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.042920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.043694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.043973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.044246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.055021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.056441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:20.072826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:20.073937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:20.074893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:20.074893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:20.074893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:20.074893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:20.074893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 629/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:24:20.079729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:20.177759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e93e-5cb6-44b3-95ca-499acb43cb20 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.186757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:20.210931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.211592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.211885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.212161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.212881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:20.213214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.214577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:20.226294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 609/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:20.231854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:20.328939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1d99-00e3-46ae-9c68-d761feb802ca None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.362036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.362915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.363195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.363465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.366068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:20.370949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.372370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:20.386561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:20.387688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:20.392890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 630/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:20.394207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:20.494571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855c656-b12a-4af9-a766-f47c4cdcbbed None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.527564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.528226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.528508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.528784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.529719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:20.530028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.534788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:20.549480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 610/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:24:20.555300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:20.653440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1381250-3443-4a19-9a9b-7556e32f0431 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.685557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.686198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.686481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.686756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.746894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:20.753795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.755234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:20.850624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 631/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:24:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:24:20.856494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:20.884152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:20.952807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbbe37-b60d-47c4-9df8-a9dd69ed3ac4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:20.985361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:20.986243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:20.986526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:20.986806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:20.996919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:20.998445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:21.017251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:21.018338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:21.019311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:21.019311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:21.019311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:21.019311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:21.019311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 611/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:21.024059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:21.116971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:21.122670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c9a3d-5ba7-4d1c-8e92-153048ebbda0 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:21.157175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:21.157833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:21.158114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:21.158385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:21.159092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:21.159394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:21.160791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:21.171664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 632/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:21.177883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:21.254835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:21.276125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bdc8-5065-40bf-af32-a4de20f27c66 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:21.309054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:21.309870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:21.310153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:21.310429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:21.317993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:21.319576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:21.333426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:21.334592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:21.335799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 612/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:21.340731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:21.442924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb135fa-02a6-4608-b9da-51a52becf5f3 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:21.476451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:21.478317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:21.478317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:21.478317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:21.478592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:21.478779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:21.480201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6060de0-1892-4512-b446-8ae08e65fc43 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:21.493010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 633/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:21.499214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:21.599061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87617d56-9b5a-4e0e-98a7-fd2f8fe7d653 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:21.632152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:21.632793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:21.633079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:21.633388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:21.634213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] RBAC: Adding query filter params (role.id=3fbeb7e9911f44ceaeffd3742d9c728c, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=e40592fc25234d65a0be99d3d29869cb) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:21.634527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:21.635945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b83e9-d518-4900-956c-5c669f552ee2 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:21.641841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 613/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:24:21 2022] GET /identity/v3/role_assignments?user.id=e40592fc25234d65a0be99d3d29869cb&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=3fbeb7e9911f44ceaeffd3742d9c728c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:23.730280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95460b-6204-4eb8-b01f-d8aded9487f4 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:23.730794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95460b-6204-4eb8-b01f-d8aded9487f4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:23.730794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95460b-6204-4eb8-b01f-d8aded9487f4 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:23.732145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 634/1247] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:23.737319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:23.737776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:23.737912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:23.742963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:23.755694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:23.759742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:23.822303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 614/1248] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:23.846514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95460b-6204-4eb8-b01f-d8aded9487f4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:23.865032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:23.944826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95460b-6204-4eb8-b01f-d8aded9487f4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:23.978755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:23.979528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:23.979807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:23.980076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:23.992865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:23.997864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:23.999043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:23.999976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 635/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:24.006108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.102288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69baac4-e8df-45da-a3ba-e2edfe327247 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.131739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:24.136493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.137164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.137464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.137744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.138484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:24.138793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:24.140170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:24.148008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 615/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:24.153778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.229119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:24.250901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159491-b558-42b5-a48b-35874a96d4e9 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.282826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.283604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.283920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.284194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.294271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:24.296165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:24.312533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:24.313669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:24.314823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:24.314823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:24.314823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:24.314823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:24.314823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 636/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:24.319984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.419451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6c698-b94e-4656-b425-2eaca948a881 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.452589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.453279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.453561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.453836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.454577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:24.454883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:24.456254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:24.467993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 616/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:24.474198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.572279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb4144-66c7-4577-b830-4fd4b2f0002b None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.597574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:24.604910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.605736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.606016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.606287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.614029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:24.615452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:24.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:24.630268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:24.631778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 637/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:24:24.636270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.691580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:24.734736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8681e-f540-4082-ad00-30b8455f21d9 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.767986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.768678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.768967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.769279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.770121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:24.770433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:24.771861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:24.788223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 617/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:24.794004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:24.900294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3baaf-6c0f-493b-b00e-941ff1d1eeb4 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:24.932701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:24.933362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:24.933647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:24.933921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:24.988104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:24.999812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.001325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.096784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 638/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:24:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:24:25.102579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.201774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09043a1-5517-4056-b965-1926f2716f9b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.214976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:25.235729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.236508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.236877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.237678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.247556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.249027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:25.266664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:25.267746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:25.268941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:25.268941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:25.268941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:25.268941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:25.268941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 618/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:25.273669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.358460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:25.370590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dee95-75a8-4c0c-ba4f-086d1111cb95 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.403057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.403715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.404002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.404286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.405002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:25.405340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.407511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.419205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 639/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:25.425403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.540608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459d29-8e5e-4821-b9b0-00a77b012a90 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.573690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.574518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.574803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.575078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.582118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.583587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:25.597699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:25.598696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:25.599715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 619/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:25.604902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.705199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73849-2d0e-45b0-b97e-b501a71f19ce None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.738950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.739594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.739875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.740148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.740928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:25.741262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.742685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13195e-cdbe-4a4d-968f-576db3f23f3c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.755140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 640/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:25.761445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.787390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:25.860999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95433c-e7dc-4e0f-94b6-a996e26c03ba None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.894647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.895292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.895572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.895847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.896669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=e40592fc25234d65a0be99d3d29869cb) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:25.896976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.898427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59e586-cb61-49e0-a094-c1a4a71b0bd8 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.904685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 620/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/role_assignments?user.id=e40592fc25234d65a0be99d3d29869cb&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:27.888103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10ec1-2109-45b4-af31-c278c234ab01 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:27.888618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10ec1-2109-45b4-af31-c278c234ab01 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:27.888618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10ec1-2109-45b4-af31-c278c234ab01 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:27.890038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 641/1261] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:27.895462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:27.895899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:27.896004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:27.912834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:27.914057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:27.979188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 621/1262] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:24:28.001961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c10ec1-2109-45b4-af31-c278c234ab01 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.013460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.099551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c10ec1-2109-45b4-af31-c278c234ab01 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.132322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.133103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.133419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.133696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.148702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:28.152548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:28.153722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:28.154636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 642/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:28.160474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.178826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.257526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922868d-2af5-4da7-9e9e-8d577ec16a8f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.290739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.291403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.291684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.292000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.292746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:28.293052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.294434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.302783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 622/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:28.308489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.374046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.407052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d7bf6-4ab4-41f3-a12d-1b2766e34767 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.439684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.440517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.440795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.441069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.451427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.452918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:28.470499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:28.472199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:28.473151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:28.473151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:28.473151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:28.473151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:28.473151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 643/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:28.478196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.559199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.577477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494b9df-d754-4418-992b-ecba40825110 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.610364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.611107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.611388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.611664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.612374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:28.612679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.614069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.625473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 623/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:28.631660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.729633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c36b-7348-4e8a-8675-847c04c1b2a9 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.750392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.764091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.764884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.765196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.765481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.773430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.774874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:28.789395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:28.790635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:28.791596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 644/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:28.796364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.893761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5098-ea6b-4e1b-97e0-a1bcd8b8b279 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.926281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.926938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.927222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.927502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.928335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:28.928648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.930105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.938498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.947130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 624/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:28.952759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:29.050859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b03e-2469-4ad7-a259-62a90dbba52c None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:29.083340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:29.084160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:29.084438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:29.084709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:29.094820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:29.096321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:29.113273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:29.114347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:29.115551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:29.115551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:29.115551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:29.115551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:29.115551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 645/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:29.120147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:29.219809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89f8d0-6cf0-46de-a7cd-9e1f7f5ca0bb None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:29.254347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:29.255006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:29.255542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:29.255817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:29.256530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:29.256835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:29.258225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:29.269191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 625/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:29.275153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:29.307374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:29.372494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108b45-acda-4165-a47e-fb78b02b9489 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:29.404375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:29.405147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:29.405458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:29.405733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:29.412643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:29.417629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:29.429276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:29.430404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:29.431630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 646/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:29.436380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:29.501507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:29.542181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46fb78-587c-49ab-9bc0-ce41efd296fc None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:29.575710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:29.576841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:29.577170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:29.577458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:29.578250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:29.578563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:29.579989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b4975-e266-4320-9c83-6b41924bd3ae None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:29.592245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 626/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:29.599066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:29.689251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:29.697642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd7c-9d05-40db-8881-8ba8d5af0cd6 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:29.730757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:29.731636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:29.731916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:29.732189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] PATH_INFO: `/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/e40592fc25234d65a0be99d3d29869cb/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:29.734185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dfc15225d38c4588bdeedd63e0cbdd95, user_id=e40592fc25234d65a0be99d3d29869cb, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:29.735606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ddf0e-f665-411f-afc6-71ab0b8b61f0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:29.841915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 647/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Jul 12 14:24:29 2022] PUT /identity/v3/projects/dfc15225d38c4588bdeedd63e0cbdd95/users/e40592fc25234d65a0be99d3d29869cb/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:24:31.842094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72d48-b17f-42b9-9512-9071f0068018 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:31.842611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72d48-b17f-42b9-9512-9071f0068018 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:31.842611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72d48-b17f-42b9-9512-9071f0068018 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:31.843924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 627/1274] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:31.849636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:31.850043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:31.850217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:31.866971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:31.868147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:31.934925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 648/1275] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:31.958054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72d48-b17f-42b9-9512-9071f0068018 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.058865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72d48-b17f-42b9-9512-9071f0068018 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.076647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:32.093634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.094450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.094741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.095022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.104754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:24:32.112459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:32.133140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:32.134302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 628/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Jul 12 14:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:32.135150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.201374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:32.218925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d9e4d-86c0-4acf-b9e0-ae863e72f33a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.251555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.252848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.253133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.253442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.254245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:32.254562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:32.256174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:32.264038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 649/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:32.269941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.368420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d4d4f-1d71-40a4-b466-744840d292c6 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.401221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.402007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.402294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.402587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.412945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:32.414418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:32.431583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:32.432708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:32.433877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:32.433877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:32.433877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:32.433877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:32.433877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 629/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:32.438632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.538395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcffa1-7734-4cf8-814f-4166d41aadf9 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.571519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.572176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.572461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.572740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.573481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:32.573790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:32.575216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:32.586649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 650/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:32.592310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.623577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:32.694376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63e878-cc24-41ad-a134-52ef1276573c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.727385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.731286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.731286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.731286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.737662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:32.739083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:24:32.753036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:32.754140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:32.755475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 630/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:32.760242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:32.764008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:32.857265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0fbae-1b97-4ae9-ab7b-f27ee61b48ca None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:32.890723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:32.891319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:32.891603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:32.891880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:32.892709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:32.893017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:32.894536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:32.911507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 651/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:32.917254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.006185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.015469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fe12-53d8-425c-a496-21f8e8402ae4 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.048015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.048775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.049066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.049365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.113990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.115517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.201716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.212004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 631/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Jul 12 14:24:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:24:33.217759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.278587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.315670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3c4e-81f2-4fc7-874e-2b55f5ff0463 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.348229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.349008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.349318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.349592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.360144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.361648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:24:33.378852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:33.380270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:33.381251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:24:33.381251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:24:33.381251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:24:33.381251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:33.381251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 652/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:33.385972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.482208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d247488-f19a-41b4-b608-29c8376a692a None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.514891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.515541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.515823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.516103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.516810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:33.517114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.519085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.529955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 632/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:33.535747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.633665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a8f6e-bca9-4fef-afb2-ecdf0419dc8a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.647622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.666795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.667609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.667897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.668175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.674760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.676203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:24:33.690463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:33.691531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: ERROR keystone.server.flask.application Jul 12 14:24:33.692798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 653/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:33.698480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.747517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.795581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa92896-4052-43ba-a4ae-1ba735f0dc4c None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.828841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.829520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.829827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.830107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.830936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:33.831251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.832677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74d95f-8d84-486b-b91d-de2c0e243204 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.844168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 633/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:33.850410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.951394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c232-8d2d-482b-bd97-a374d90a7a93 None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.984990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.985715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.985999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.986782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.987608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] RBAC: Adding query filter params (role.id=e92eaebf304e4ec6afef99f44d30b06d, scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95, user.id=e40592fc25234d65a0be99d3d29869cb) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:33.987918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.989368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ca827-f021-418b-b1e6-874b756792b0 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.995112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 654/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/role_assignments?user.id=e40592fc25234d65a0be99d3d29869cb&scope.project.id=dfc15225d38c4588bdeedd63e0cbdd95&role.id=e92eaebf304e4ec6afef99f44d30b06d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:36.022739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef06518-a874-4e91-89de-40b05bad385f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.023267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef06518-a874-4e91-89de-40b05bad385f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.023361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef06518-a874-4e91-89de-40b05bad385f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.024764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 634/1288] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:36.029708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.030172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.030268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.047069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:36.048231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:36.077309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:36.113721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 655/1289] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:36.136829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef06518-a874-4e91-89de-40b05bad385f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:36.235315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef06518-a874-4e91-89de-40b05bad385f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:36.248206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:36.276225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:36.276962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.277283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.277562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.278217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:36.279584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:36.291274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:36.292546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:36.293666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 635/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:24:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:36.298933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:36.400581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350f2f5-0920-4766-be7f-801b12c037d9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:36.435409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:36.436067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.436345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.436614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.437347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:36.437653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:36.439030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e129862-ea87-4adf-bd9c-4be96036b736 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:36.443917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 656/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Jul 12 14:24:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:36.449710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:36.549802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380bdcd-8bdb-4526-8464-ff16273e31aa None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:36.582485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:36.583136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.583417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.583692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] PATH_INFO: `/v3/services` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.584387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:36.584691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:36.586231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836e6b8-019b-485a-815b-57c5d7a0baa6 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:36.590676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 636/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Jul 12 14:24:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:38.607189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2219b-41be-45d3-8cf3-c9dc3ebca101 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:38.607717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2219b-41be-45d3-8cf3-c9dc3ebca101 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:38.607717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2219b-41be-45d3-8cf3-c9dc3ebca101 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:38.609089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 657/1293] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:38.614685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633851d2-7b13-4d7b-ab07-f946221e9227 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:38.615107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633851d2-7b13-4d7b-ab07-f946221e9227 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:38.615253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633851d2-7b13-4d7b-ab07-f946221e9227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:38.631951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-633851d2-7b13-4d7b-ab07-f946221e9227 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:38.633131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-633851d2-7b13-4d7b-ab07-f946221e9227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:38.662016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:38.697731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 637/1294] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:38.719960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2219b-41be-45d3-8cf3-c9dc3ebca101 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:38.751222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:38.819237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2219b-41be-45d3-8cf3-c9dc3ebca101 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:38.851530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:38.852180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:38.852461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:38.852730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:38.853480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:38.854830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767fbfa-a045-4fd4-8246-a7e01c07aa20 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:38.934070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 658/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Jul 12 14:24:38 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:24:40.955505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3882e69-d955-45be-8c35-efa45f6cc7b0 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:40.956028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3882e69-d955-45be-8c35-efa45f6cc7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:40.956152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3882e69-d955-45be-8c35-efa45f6cc7b0 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:40.957477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 638/1296] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:40.962708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:40.963115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:40.963258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:40.979483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:40.980705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:41.050624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 659/1297] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:24:41.073051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3882e69-d955-45be-8c35-efa45f6cc7b0 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:41.170756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3882e69-d955-45be-8c35-efa45f6cc7b0 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:41.203519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:41.205363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:41.206157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:41.206386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:41.207039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:41.208404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:41.220241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:41.221315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:41.222318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 639/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:24:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:41.227747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:41.246756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:41.325096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286044b6-bea9-4a4a-8d37-72fb8b3c628a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:41.357246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:41.357898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:41.358179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:41.358452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:41.359147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:41.359450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:41.361335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:41.366395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 660/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Jul 12 14:24:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:41.372278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:41.403646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:41.471475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a355-b065-486b-9688-77f83ca41b5b None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:41.504611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:41.505542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:41.505824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:41.506098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:41.506873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] RBAC: Adding query filter params (interface=public, service_id=ea5445a7686a43638bcc41ec83a3fd06, region_id=RegionOne) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:41.507184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:41.508533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113a552-09cb-44c5-ab5d-a7842c37e8dd None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:41.513563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 640/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Jul 12 14:24:41 2022] GET /identity/v3/endpoints?service_id=ea5445a7686a43638bcc41ec83a3fd06&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:41.519349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:41.598458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:41.616178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b1168-7ddb-498b-a627-a3e1da30732a None admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:41.648327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:41.648984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:41.649302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:41.649589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:41.650268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:41.651634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748d2d0-ac8d-497a-9517-4ab34b655b5e None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:41.659759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 661/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Jul 12 14:24:41 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:24:43.637740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508eeed-faa8-415c-996f-dba6ff8950d5 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:43.638301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508eeed-faa8-415c-996f-dba6ff8950d5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:43.638301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508eeed-faa8-415c-996f-dba6ff8950d5 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:43.639716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 641/1302] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:43.645510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:43.645896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:43.646341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:43.662582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:43.663760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:43.729661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 662/1303] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:24:43.752235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508eeed-faa8-415c-996f-dba6ff8950d5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:43.845637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:43.852082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508eeed-faa8-415c-996f-dba6ff8950d5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:43.884551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:43.885372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:43.885656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:43.885930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:43.886571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:43.887960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:24:43.900331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:24:43.902741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application with Lock( Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: ERROR keystone.server.flask.application Jul 12 14:24:43.903731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 642/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Jul 12 14:24:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:24:43.907539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:43.983124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:44.005539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cde1a-ff3d-43c1-82ea-9c5f8c72626f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:44.037912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:44.038567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:44.038852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:44.039131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] PATH_INFO: `/v3/services` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:44.039910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:24:44.040220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:44.041633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368230f6-f806-4c1d-8c12-6a440a5b03f7 None admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:44.048154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 663/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Jul 12 14:24:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:24:44.053870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:44.154019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951f24-9f8a-4e75-8e9d-feb2c35418b3 None admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:44.186019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:44.186660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:44.186938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:44.187211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:44.187984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:44.189866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7c363-0c9a-4943-b9b8-612194c16060 None admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:44.263868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 643/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Jul 12 14:24:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:24:54.058175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=30 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:54.259094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:46.193125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65973c3-4ef3-4f1c-bcc8-aeab2c1e014f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:46.193682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65973c3-4ef3-4f1c-bcc8-aeab2c1e014f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:46.193773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65973c3-4ef3-4f1c-bcc8-aeab2c1e014f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:46.200464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc57d1-87d9-4421-8350-eb5047936575 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:46.200931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc57d1-87d9-4421-8350-eb5047936575 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:46.201036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc57d1-87d9-4421-8350-eb5047936575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:46.205598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 664/1307] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:30:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:46.278813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-eecc57d1-87d9-4421-8350-eb5047936575 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:46.280026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-eecc57d1-87d9-4421-8350-eb5047936575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:46.421800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 644/1308] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:46.475828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:46.476226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:46.476609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:46.478546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 665/1309] 10.222.0.30 () {54 vars in 964 bytes} [Tue Jul 12 14:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:46.491743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d2e1-6cb7-4795-a658-462e05e459ad None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:46.492044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d2e1-6cb7-4795-a658-462e05e459ad None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:46.492330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d2e1-6cb7-4795-a658-462e05e459ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:46.663331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-3685d2e1-6cb7-4795-a658-462e05e459ad None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:46.664710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3685d2e1-6cb7-4795-a658-462e05e459ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:46.689250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:46.758477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 645/1310] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Jul 12 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:46.768183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:46.770482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:46.899164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad494b-6819-4302-8eb2-712b5914e7d8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:46.980143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:46.981214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:46.981749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:46.982253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:46.984117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:47.042821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:47.099807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:47.101641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e498c2f-92d5-4a36-bd89-1fadb92adc40 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:47.133562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 666/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:49.527821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:49.528370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:49.528370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:49.529790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 646/1312] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:49.535117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3659-86db-447f-bdaf-c66b1d082fb0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:49.535526 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3659-86db-447f-bdaf-c66b1d082fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:49.535684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3659-86db-447f-bdaf-c66b1d082fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:49.618103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8baa3659-86db-447f-bdaf-c66b1d082fb0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:49.619297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8baa3659-86db-447f-bdaf-c66b1d082fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:49.718131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 667/1313] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:49.754979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:49.757047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:49.869177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4daa9-4d7b-4928-a2e2-9af9cbf3f194 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:49.945827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:49.948395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:49.949043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:49.949352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:49.949629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:49.951250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:50.060281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:50.061689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a045d-bd90-4ac3-ae6c-b155b3488a2c service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:50.082453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 647/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:52.572199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:52.572726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:52.572726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:52.574155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 668/1315] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:52.579354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1176d0-69ae-4eaa-9e97-1183b7015fd4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:52.579774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1176d0-69ae-4eaa-9e97-1183b7015fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:52.580207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1176d0-69ae-4eaa-9e97-1183b7015fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:52.645529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-fa1176d0-69ae-4eaa-9e97-1183b7015fd4 None None] MFA Rules not processed for user `fa38071580e54275a3c6d76058090878`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:52.647420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fa1176d0-69ae-4eaa-9e97-1183b7015fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:52.698350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:52.742194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 648/1316] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:52.782130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:52.783651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:52.793563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:52.887263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a22d6-daba-43e4-a2f3-1c312dc812cb None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:52.960993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:52.961591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:52.961879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:52.962157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:52.963852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:53.026542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:53.072477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:53.073996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f49be3-c6fb-4409-b9d4-9d62fd33fc23 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:53.091915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 669/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:55.930437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:55.930963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:55.930963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:55.932363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 649/1318] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:55.937577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99c4a7-3749-48ae-9eb5-ee106b05fb55 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:55.938004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99c4a7-3749-48ae-9eb5-ee106b05fb55 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:55.938122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99c4a7-3749-48ae-9eb5-ee106b05fb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:56.001233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4d99c4a7-3749-48ae-9eb5-ee106b05fb55 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:56.002421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4d99c4a7-3749-48ae-9eb5-ee106b05fb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:56.095242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 670/1319] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:56.137657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:56.139204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:56.240933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cae04e-5e0d-4911-bf8b-d1c628d41e43 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:56.310801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:56.311441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:56.311723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:56.312000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:56.313635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:56.322280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:56.420872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:56.422343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138250ff-ac80-4fa6-ab42-a41d7b837ac2 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:56.442185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 650/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:59.428510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b702-2729-40a3-837f-73044d030cbd None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:59.429075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b702-2729-40a3-837f-73044d030cbd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:59.429075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1b702-2729-40a3-837f-73044d030cbd None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:59.430689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 671/1321] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:59.435665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386228e1-a078-49be-9a9b-86a2f2b95cde None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:59.436088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386228e1-a078-49be-9a9b-86a2f2b95cde None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:59.436275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386228e1-a078-49be-9a9b-86a2f2b95cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:59.502139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-386228e1-a078-49be-9a9b-86a2f2b95cde None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:59.503379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-386228e1-a078-49be-9a9b-86a2f2b95cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:59.561135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:59.601562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 651/1322] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:59.651423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:59.651710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:59.651986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:59.653407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 672/1323] 10.222.0.30 () {54 vars in 964 bytes} [Tue Jul 12 14:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:59.659659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79343-a004-4520-9a91-85520078de05 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:59.659955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79343-a004-4520-9a91-85520078de05 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:59.660237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79343-a004-4520-9a91-85520078de05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:59.741288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f4b79343-a004-4520-9a91-85520078de05 None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:59.742520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f4b79343-a004-4520-9a91-85520078de05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:59.821949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 652/1324] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Jul 12 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:59.830902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:59.832465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:59.851497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:59.935575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261561-157d-4bdd-b041-d6d131bd6fae None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:00.007811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:00.008354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:00.008636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:00.008909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:00.010595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:00.081459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:00.119324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:00.120760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a55f3-3041-4bd7-ad93-78236f6c4bb1 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:00.138623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 673/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:02.424065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:02.424637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:02.424743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:02.426308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 653/1326] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:02.434242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b1d5-2a79-4ea7-a319-4bfa03715af9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:02.434700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b1d5-2a79-4ea7-a319-4bfa03715af9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:02.434805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b1d5-2a79-4ea7-a319-4bfa03715af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:02.506721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-efc3b1d5-2a79-4ea7-a319-4bfa03715af9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:02.507961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-efc3b1d5-2a79-4ea7-a319-4bfa03715af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:02.604305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 674/1327] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:02.639541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:02.641148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:02.712086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:02.754267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c767f4-0091-43cb-9ce1-298e7ce8f0b5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:02.827659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:02.828588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:02.828864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:02.829135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:02.830809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:02.936542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:02.939390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:02.940771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d517151-0ac8-4aab-a29a-e03155517413 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:02.959114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 654/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:05.126047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:05.126522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:05.126639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:05.127992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 675/1329] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:05.133830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb15b5-26c4-480b-be00-c7f1728118d9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:05.134239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb15b5-26c4-480b-be00-c7f1728118d9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:05.134387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb15b5-26c4-480b-be00-c7f1728118d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:05.198229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d7cb15b5-26c4-480b-be00-c7f1728118d9 None None] MFA Rules not processed for user `fa38071580e54275a3c6d76058090878`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:05.199432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d7cb15b5-26c4-480b-be00-c7f1728118d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:05.311649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 655/1330] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:05.349890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:05.351422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:05.385621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:05.457701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260796e-49fa-4ee7-84cc-d9495e3340cd None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:05.529365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:05.529948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:05.530232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:05.530512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:05.532625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:05.608156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:05.641003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:05.642435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ef09-0a0d-4e9a-8760-3ef997b35138 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:05.660538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 676/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:08.250515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:08.251015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:08.251148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:08.252719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 656/1332] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:08.258561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cda5-427e-45cf-8173-8d64b19b8c5a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:08.258996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cda5-427e-45cf-8173-8d64b19b8c5a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:08.259128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cda5-427e-45cf-8173-8d64b19b8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:08.323765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d4d0cda5-427e-45cf-8173-8d64b19b8c5a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:08.324965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d4d0cda5-427e-45cf-8173-8d64b19b8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:08.422654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 677/1333] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:08.458554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:08.460111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:08.529577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:08.563021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c858aa9-9b60-4613-a88b-3b87cba7c32f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:08.634635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:08.635274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:08.635557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:08.635832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:08.637739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:08.743849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:08.745231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e616405-d279-4aa8-a445-1f0162b3310b service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:08.754924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:08.763164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 657/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:10.809469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:10.809974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:10.809974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:10.811326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 678/1335] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:10.816359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa50aa0-a4ce-4f28-b1b9-719eb540a926 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:10.816760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa50aa0-a4ce-4f28-b1b9-719eb540a926 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:10.816958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa50aa0-a4ce-4f28-b1b9-719eb540a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:10.896920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1fa50aa0-a4ce-4f28-b1b9-719eb540a926 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:10.898171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1fa50aa0-a4ce-4f28-b1b9-719eb540a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:10.992462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 658/1336] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:11.031734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:11.033312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:11.072704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:11.136253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7046d-b73d-4b39-a1bd-df94580da36a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:11.213715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:11.214361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:11.214645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:11.214921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:11.216666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:11.297439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:11.323033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:11.324398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930c43d-7708-45eb-9702-aab5c872a6db service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:11.342263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 679/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:13.920047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:13.920627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:13.920627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:13.922058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 659/1338] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:13.927446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c982-83bd-4f16-9415-14548cf58eb9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:13.927854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c982-83bd-4f16-9415-14548cf58eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:13.927996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c982-83bd-4f16-9415-14548cf58eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:13.992711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8c09c982-83bd-4f16-9415-14548cf58eb9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:13.993965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8c09c982-83bd-4f16-9415-14548cf58eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:14.088406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 680/1339] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:31:14.123728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:14.125290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:14.200712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:14.227575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ec14-b527-4083-9fc9-ecf5e131f30b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:14.299327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:14.299969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:14.300252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:14.300558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:14.302770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:14.409714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:14.411662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a4bcb-b77c-4e7d-a295-2ab981a1f7f8 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:14.430930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 660/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:18.597072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:18.597624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:18.597732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:18.599059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 681/1341] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:18.604691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536583f-e986-4b0c-994b-ba0c24d8a6f1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:18.605076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536583f-e986-4b0c-994b-ba0c24d8a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:18.605694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536583f-e986-4b0c-994b-ba0c24d8a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:18.610069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:18.674532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5536583f-e986-4b0c-994b-ba0c24d8a6f1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:18.675955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5536583f-e986-4b0c-994b-ba0c24d8a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:18.779474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 661/1342] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:18.820542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:18.822237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:18.872394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:18.929353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7b25-af5f-4153-b152-63afb20146ec None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:19.002381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:19.003054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:19.003342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:19.003621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:19.005351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:19.098875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:19.115778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:19.117257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cec0b-191b-4d7a-85e7-6c345e8eb0fb service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:19.135711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 682/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:22.138958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:22.139514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:22.139514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:22.140894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 662/1344] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:22.146284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ad32-ee11-483b-b47e-266ece957c41 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:22.146713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ad32-ee11-483b-b47e-266ece957c41 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:22.146830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ad32-ee11-483b-b47e-266ece957c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:22.213995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-af73ad32-ee11-483b-b47e-266ece957c41 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:22.215211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-af73ad32-ee11-483b-b47e-266ece957c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:22.308961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 683/1345] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:22.345726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:22.347245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:22.435114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:22.451889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e5c81-c713-4a50-bd96-7697ec1587b9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:22.526504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:22.527158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:22.527442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:22.527718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:22.529378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:22.662046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:22.663479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200972a-0391-496b-8c4a-d1e7358974e5 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:22.682072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 663/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:26.853734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:26.854269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:26.854269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:26.855608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 684/1347] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:26.861260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee158-7bec-4558-ae06-d245a56130c3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:26.861741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee158-7bec-4558-ae06-d245a56130c3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:26.861839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee158-7bec-4558-ae06-d245a56130c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:26.877050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:26.931756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-955ee158-7bec-4558-ae06-d245a56130c3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:26.934064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-955ee158-7bec-4558-ae06-d245a56130c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:27.034292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 664/1348] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:27.071883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:27.073467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:27.133258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:27.180090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871d39-eb80-4f29-9b5b-ae511742f5f8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:27.256053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:27.256728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:27.257013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:27.257424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:27.259108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:27.365093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:27.368578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:27.370064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80054391-5a8b-4330-b4e3-62bac6892206 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:27.388594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 685/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:32.251007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08968c-745d-4501-85b0-5e52f6d6fb00 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:32.251560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08968c-745d-4501-85b0-5e52f6d6fb00 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:32.251560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08968c-745d-4501-85b0-5e52f6d6fb00 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:32.253114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 665/1350] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:32.265969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3986-3227-4185-af07-e90ea62d1e50 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:32.266853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3986-3227-4185-af07-e90ea62d1e50 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:32.267287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd3986-3227-4185-af07-e90ea62d1e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:32.354864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-c2dd3986-3227-4185-af07-e90ea62d1e50 None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:32.356290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c2dd3986-3227-4185-af07-e90ea62d1e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:32.480588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 686/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:31:36.636537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765e21-d602-402b-89f1-e6b2b20b73f6 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:36.637412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765e21-d602-402b-89f1-e6b2b20b73f6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:36.637722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7765e21-d602-402b-89f1-e6b2b20b73f6 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:36.639271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 666/1352] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:36.644159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bf5-b9ed-497d-b96b-18d5de1321cc None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:36.645507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bf5-b9ed-497d-b96b-18d5de1321cc None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:36.646002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee38bf5-b9ed-497d-b96b-18d5de1321cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:36.704608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:36.731172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dee38bf5-b9ed-497d-b96b-18d5de1321cc None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:36.733218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dee38bf5-b9ed-497d-b96b-18d5de1321cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:36.857693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 687/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:37.032176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=59 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:38.277757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba8aa-8ae5-4823-96d5-edc418149a21 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:38.278385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba8aa-8ae5-4823-96d5-edc418149a21 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:38.278385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba8aa-8ae5-4823-96d5-edc418149a21 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:38.279894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 667/1354] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:38.285049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c98dd-cb33-4f06-81e7-879325a31da6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:38.285990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c98dd-cb33-4f06-81e7-879325a31da6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:38.286169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c98dd-cb33-4f06-81e7-879325a31da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:38.360817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dd8c98dd-cb33-4f06-81e7-879325a31da6 None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:38.362039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dd8c98dd-cb33-4f06-81e7-879325a31da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:38.404235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:38.479448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 688/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:41.844825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9bd9-8098-470f-bc4b-e4870d4fd396 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:41.845386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9bd9-8098-470f-bc4b-e4870d4fd396 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:41.845472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c9bd9-8098-470f-bc4b-e4870d4fd396 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:41.847877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 668/1356] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:41.853856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a25c40-975c-40ca-ba23-7045d7b7bad1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:41.858067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a25c40-975c-40ca-ba23-7045d7b7bad1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:41.858730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a25c40-975c-40ca-ba23-7045d7b7bad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:41.973233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f7a25c40-975c-40ca-ba23-7045d7b7bad1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:41.974755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f7a25c40-975c-40ca-ba23-7045d7b7bad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:42.087350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 689/1357] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:42.148471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:42.148994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:42.149458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:42.151007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 669/1358] 10.222.0.30 () {54 vars in 961 bytes} [Tue Jul 12 14:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:42.166655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a015d0-0bcd-4ece-b39f-ea70958245f3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:42.167192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a015d0-0bcd-4ece-b39f-ea70958245f3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:42.167644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a015d0-0bcd-4ece-b39f-ea70958245f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:42.183702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:42.251440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-06a015d0-0bcd-4ece-b39f-ea70958245f3 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:42.252547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-06a015d0-0bcd-4ece-b39f-ea70958245f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:42.375244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 690/1359] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Jul 12 14:31:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:42.383951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:42.385966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:42.551319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358244bb-5228-4030-9add-2b7462bf3f0e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:42.630582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:42.673671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:42.674500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:42.675052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:42.675892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:42.677841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:42.774557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:42.776429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb09e6-227b-467f-a65c-77507e1fa8a5 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:42.797224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 670/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:45.013901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:45.014479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:45.014479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:45.016420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 691/1361] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:45.022524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57571cf-7b3b-4591-b78b-d6148d2d630a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:45.022524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57571cf-7b3b-4591-b78b-d6148d2d630a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:45.022999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57571cf-7b3b-4591-b78b-d6148d2d630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:45.092149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a57571cf-7b3b-4591-b78b-d6148d2d630a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:45.093434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a57571cf-7b3b-4591-b78b-d6148d2d630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:45.132765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:45.196576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 671/1362] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:45.241506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:45.243022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:45.318995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:45.408093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12efb56-3df3-4782-be23-de98f436cef5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:45.584477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:45.585141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:45.585475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:45.585754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:45.587412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:45.611790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:45.687398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:45.689005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c28fb-a8d6-49fe-a97e-9b6567eb1f5d service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:45.708323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 692/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:48.149606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dddea-f3dd-4543-a4dc-020d8afb5bb1 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:48.150171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dddea-f3dd-4543-a4dc-020d8afb5bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:48.150171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dddea-f3dd-4543-a4dc-020d8afb5bb1 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:48.151547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 672/1364] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:48.156928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d567-baf8-4160-ae95-770d5ac4087f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:48.157413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d567-baf8-4160-ae95-770d5ac4087f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:48.157630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5d567-baf8-4160-ae95-770d5ac4087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:48.231545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bec5d567-baf8-4160-ae95-770d5ac4087f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:48.233324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bec5d567-baf8-4160-ae95-770d5ac4087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:48.340295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:48.345264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 693/1365] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:48.391988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:48.392337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:48.392854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:48.394986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 673/1366] 10.222.0.30 () {54 vars in 961 bytes} [Tue Jul 12 14:31:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:48.400765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e515-e223-4337-ac73-1e905ed8d4a8 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:48.401332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e515-e223-4337-ac73-1e905ed8d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:48.401620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e515-e223-4337-ac73-1e905ed8d4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:48.486265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bd86e515-e223-4337-ac73-1e905ed8d4a8 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:48.487479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bd86e515-e223-4337-ac73-1e905ed8d4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:48.625383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 694/1367] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Jul 12 14:31:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:48.634146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:48.636570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:48.790631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d950-66d6-40d3-b8a5-e89b12c8e09b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:48.812012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:48.910499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:48.911075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:48.911361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:48.911636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:48.913353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:49.008687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:49.010360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cbbcb-f1a2-412f-8b97-f5548d81f67e service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:49.030053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 674/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:49.957296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a592c-bd86-4bde-9a56-aa0db2f5ffac None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:49.958175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a592c-bd86-4bde-9a56-aa0db2f5ffac None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:49.958175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a592c-bd86-4bde-9a56-aa0db2f5ffac None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:49.959380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 695/1369] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:49.964529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ef9-05b1-4e74-90b8-8d1b4960f974 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:49.964942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ef9-05b1-4e74-90b8-8d1b4960f974 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:49.965115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ef9-05b1-4e74-90b8-8d1b4960f974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:49.996518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:50.031181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7eaf1ef9-05b1-4e74-90b8-8d1b4960f974 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:50.032383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7eaf1ef9-05b1-4e74-90b8-8d1b4960f974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:50.134942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 675/1370] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:50.478174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:50.478174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:50.478174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:50.480765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 696/1371] 10.222.0.30 () {54 vars in 963 bytes} [Tue Jul 12 14:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:50.484406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b3823-3b7f-4caf-b6bf-5f657148ef74 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:50.484693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b3823-3b7f-4caf-b6bf-5f657148ef74 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:50.484968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b3823-3b7f-4caf-b6bf-5f657148ef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:50.562583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-cf6b3823-3b7f-4caf-b6bf-5f657148ef74 None None] MFA Rules not processed for user `177ef85784a94f3782aa48b60a6fb8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:50.563776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cf6b3823-3b7f-4caf-b6bf-5f657148ef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:50.615448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:50.646015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 676/1372] 10.222.0.30 () {58 vars in 1060 bytes} [Tue Jul 12 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:50.653580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:50.655237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:50.709573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:50.768695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80ec7e-ef8b-4d71-9cb9-852d19872565 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:50.841815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:50.842469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:50.842754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:50.843032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:50.844807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:50.942903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:50.956519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:50.958012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068045b5-dcdb-47e5-9993-d0f6ae394ec2 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:50.977014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 697/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:51.242957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:51.243419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:51.243671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:51.245289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 677/1374] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:51.250071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a16a2-73e7-418e-a8f4-50ec6c9b1f90 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:51.250549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a16a2-73e7-418e-a8f4-50ec6c9b1f90 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:51.250549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a16a2-73e7-418e-a8f4-50ec6c9b1f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:51.318392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-109a16a2-73e7-418e-a8f4-50ec6c9b1f90 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:51.319592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-109a16a2-73e7-418e-a8f4-50ec6c9b1f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:51.434295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 698/1375] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:51.468129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:51.473679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:51.620239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ef0d7-9456-424c-ad80-de446f2ca627 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:51.673116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:51.733653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:51.734382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:51.734663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:51.734934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:51.736574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:51.825961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:51.827549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d0b9a-152e-412d-80d3-4e7767fee64b service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:51.845734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 678/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:54.045275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:54.046470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:54.046976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:54.048566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 699/1377] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:54.054743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ca55-a36e-479b-8a0b-911a69d1a5ef None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:54.055587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ca55-a36e-479b-8a0b-911a69d1a5ef None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:54.057709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ca55-a36e-479b-8a0b-911a69d1a5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:54.133318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f5e2ca55-a36e-479b-8a0b-911a69d1a5ef None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:54.134515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f5e2ca55-a36e-479b-8a0b-911a69d1a5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:54.141681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:54.231900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 679/1378] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:31:54.277731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:54.280086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:54.349427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:54.446044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49f8a4-b805-4313-bc24-71881118f70e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:54.613349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:54.613943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:54.614237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:54.614528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:54.616222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:54.640326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:54.714676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:54.716290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921658-8f84-4ef1-8c42-173995c73879 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:54.736104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 700/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:31:56.914582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:56.915385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:56.915385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:56.920437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 680/1380] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:31:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:31:56.921875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68cd-9dcb-4ed3-ad1f-cd8aea14d5f4 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:56.922411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68cd-9dcb-4ed3-ad1f-cd8aea14d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:56.922607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f68cd-9dcb-4ed3-ad1f-cd8aea14d5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:56.990844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-af3f68cd-9dcb-4ed3-ad1f-cd8aea14d5f4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:31:56.992059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-af3f68cd-9dcb-4ed3-ad1f-cd8aea14d5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:57.086891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:57.093668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 701/1381] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:31:57.139179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:31:57.140700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:57.286308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:57.297683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9a5fb-6bab-4ce7-be38-e0eea4049272 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:31:57.417953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:31:57.418516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:31:57.418803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:31:57.419078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:31:57.420713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:31:57.524171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:31:57.525933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d7d7f-a2dc-48e5-ad52-6e0807cadbd7 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:31:57.539632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:31:57.550816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 681/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:00.059770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:00.060315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:00.060315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:00.061785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 702/1383] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:00.068584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497fa08-c4ae-4ffb-9d0b-1b7372dd421a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:00.069010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497fa08-c4ae-4ffb-9d0b-1b7372dd421a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:00.069147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497fa08-c4ae-4ffb-9d0b-1b7372dd421a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:00.136581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4497fa08-c4ae-4ffb-9d0b-1b7372dd421a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:00.137869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4497fa08-c4ae-4ffb-9d0b-1b7372dd421a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:00.236611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 682/1384] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:00.279755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:00.281331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:00.457282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e453a-fece-4fb9-9afe-39ea3ac21f76 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:00.526197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:00.582218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:00.582875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:00.583169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:00.583445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:00.585176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:00.678990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:00.680620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090a2cc-d299-445f-a7c4-6ab191d94df9 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:00.699940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 703/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:02.800360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:02.800958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:02.800958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:02.802399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 683/1386] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:02.810573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54968-f25a-40a8-b895-25301a58bd70 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:02.811045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54968-f25a-40a8-b895-25301a58bd70 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:02.811533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c54968-f25a-40a8-b895-25301a58bd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:02.904996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-29c54968-f25a-40a8-b895-25301a58bd70 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:02.906656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-29c54968-f25a-40a8-b895-25301a58bd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:02.948824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:03.025571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 704/1387] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:03.073180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:03.075040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:03.177128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:03.254783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7a7fe-fe28-4aa2-8c00-5bbdaa80c882 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:03.392068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:03.392888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:03.393841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:03.394248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:03.396679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:03.447142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:03.501256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:03.503516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd919-1cef-4b87-8fe3-f5dfb9e67ade service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:03.523406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 684/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:05.698960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:05.699470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:05.699470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:05.700989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 705/1389] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:05.706627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0197f-941b-490e-9a2b-316adcdf0f63 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:05.707045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0197f-941b-490e-9a2b-316adcdf0f63 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:05.707191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0197f-941b-490e-9a2b-316adcdf0f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:05.792495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ceb0197f-941b-490e-9a2b-316adcdf0f63 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:05.793847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ceb0197f-941b-490e-9a2b-316adcdf0f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:05.892522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 685/1390] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:05.936563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:05.938111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.084211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736614-1d9e-4004-9a3a-1c6ede392026 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:06.102096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:06.198340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:06.198985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.199267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.199570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.201285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.288771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35621ab-d5d2-47ff-a39b-fd36abe8127f None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.289057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35621ab-d5d2-47ff-a39b-fd36abe8127f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.289371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35621ab-d5d2-47ff-a39b-fd36abe8127f None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.289702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:06.290813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 686/1391] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:32:06.291318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638d749-438e-4334-829c-b84fb3299e88 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:06.295655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc220-88e6-42dd-b939-2336f8e7314e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.296005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc220-88e6-42dd-b939-2336f8e7314e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.296346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc220-88e6-42dd-b939-2336f8e7314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.310939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:06.311108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 706/1392] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:06.361492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9e5cc220-88e6-42dd-b939-2336f8e7314e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:06.363187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9e5cc220-88e6-42dd-b939-2336f8e7314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.467002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 687/1393] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:06.522771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.523162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.523566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.525208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 707/1394] 10.222.0.30 () {54 vars in 962 bytes} [Tue Jul 12 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:06.530784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f370b-9d0c-4519-b65f-7cb50779482a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.531082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f370b-9d0c-4519-b65f-7cb50779482a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.531367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f370b-9d0c-4519-b65f-7cb50779482a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.616557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-803f370b-9d0c-4519-b65f-7cb50779482a None None] MFA Rules not processed for user `401f40d749db4643824d38db0cf533cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:06.617729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-803f370b-9d0c-4519-b65f-7cb50779482a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.656284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:06.701000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 688/1395] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Jul 12 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:06.708578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:06.711075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.733871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:06.817390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab60a9-5274-4b07-9856-480859a9ae9a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:06.892286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:06.892903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:06.893196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:06.893473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:06.895145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:06.969400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:07.005142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:07.006661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637d347-420d-46c1-a5fb-e9b29f60494b service glance] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:07.025138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 708/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:07.886942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b405a-e0f1-435a-98ba-dc1324ce8aa1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:07.887414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b405a-e0f1-435a-98ba-dc1324ce8aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:07.887541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b405a-e0f1-435a-98ba-dc1324ce8aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:07.952505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ef3b405a-e0f1-435a-98ba-dc1324ce8aa1 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:07.953788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ef3b405a-e0f1-435a-98ba-dc1324ce8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.043726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 689/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:08.096161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.096664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.096770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.098132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 709/1398] 10.222.0.30 () {54 vars in 942 bytes} [Tue Jul 12 14:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:08.104614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6e6a-2ad3-4675-9d79-f2858f8dfd85 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.105010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6e6a-2ad3-4675-9d79-f2858f8dfd85 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.105426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b6e6a-2ad3-4675-9d79-f2858f8dfd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.164246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:08.182774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-321b6e6a-2ad3-4675-9d79-f2858f8dfd85 None None] MFA Rules not processed for user `59997304919b432184f842b10ab9bba8`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:08.184008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-321b6e6a-2ad3-4675-9d79-f2858f8dfd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.291995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 690/1399] 10.222.0.30 () {58 vars in 1039 bytes} [Tue Jul 12 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:08.302131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:08.303675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.443165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ac10a-de60-498a-81ac-fe2d115c9ea6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:08.483856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:08.515470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd94a-d96a-47c0-aa63-05941a271fb2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.515762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd94a-d96a-47c0-aa63-05941a271fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.516050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fd94a-d96a-47c0-aa63-05941a271fb2 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.517466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 691/1400] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:32:08.522396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724905e-0a99-4974-b540-0219d5a675b7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.522689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724905e-0a99-4974-b540-0219d5a675b7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.522971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724905e-0a99-4974-b540-0219d5a675b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.546034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:08.546680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.546969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.547238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.548909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.588470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6724905e-0a99-4974-b540-0219d5a675b7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:08.589695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6724905e-0a99-4974-b540-0219d5a675b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.632268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:08.634054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:08.642440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 710/1401] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:32:08.649802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:08.692492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 692/1402] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:08.736304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:08.738084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:08.806006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:08.828207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7930de-cb64-432d-9352-9b98ff3298fc service swift] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:08.940290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:08.941055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:08.941365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:08.941640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:08.943277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:09.040723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:09.042303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9205efb-9bb3-4ebb-a231-c56daaa56b4c service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:09.047741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:09.067727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 711/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:11.274380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:11.274952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:11.275337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:11.278060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 693/1404] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:11.290970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7c95-9fbf-4907-a394-2213479467dc None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:11.290970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7c95-9fbf-4907-a394-2213479467dc None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:11.290970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7c95-9fbf-4907-a394-2213479467dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:11.363968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9b0a7c95-9fbf-4907-a394-2213479467dc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:11.368441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9b0a7c95-9fbf-4907-a394-2213479467dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:11.469236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 712/1405] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:11.513636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:11.515155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:11.682019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895a17d-a98f-443b-950b-932413a9e9b7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:11.742316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:11.820252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:11.820911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:11.821226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:11.821505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:11.823212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:11.928064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:11.929703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161edf3-a003-4736-b85c-cbca76b0cfa0 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:11.967358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 694/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:14.225009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:14.225975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:14.225975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:14.231075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 713/1407] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:32:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:14.232704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4494-648e-420c-a99f-dec2a57346a3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:14.233172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4494-648e-420c-a99f-dec2a57346a3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:14.233365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4494-648e-420c-a99f-dec2a57346a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:14.313735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:14.324476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5a3f4494-648e-420c-a99f-dec2a57346a3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:14.325894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5a3f4494-648e-420c-a99f-dec2a57346a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:14.463445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 695/1408] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:14.511824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:14.513813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:14.601572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:14.697637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8891e-1be3-40a4-9924-50a0b6793fff None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:14.841471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:14.842054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:14.842341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:14.842609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:14.844257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:14.888597 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:14.960004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:14.961833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9980d-c77f-4f66-b3b9-76252caf17bc service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:14.984209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 714/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:24.461388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=44 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:24.981762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=31 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:32.616547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3738-2f45-4285-b475-0fc95bc1e16d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.617098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3738-2f45-4285-b475-0fc95bc1e16d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.617098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3738-2f45-4285-b475-0fc95bc1e16d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.618415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 696/1410] 10.222.0.30 () {54 vars in 918 bytes} [Tue Jul 12 14:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:32.623379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f245-aa66-46e8-b62e-0c5ef6f00510 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.623784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f245-aa66-46e8-b62e-0c5ef6f00510 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.623927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f245-aa66-46e8-b62e-0c5ef6f00510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.697335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-e103f245-aa66-46e8-b62e-0c5ef6f00510 None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:32.698556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e103f245-aa66-46e8-b62e-0c5ef6f00510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:32.801282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 715/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Jul 12 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:32.815791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cc231-6438-4c37-9f8b-8a31c2529ef0 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.816078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cc231-6438-4c37-9f8b-8a31c2529ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.816352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cc231-6438-4c37-9f8b-8a31c2529ef0 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.817745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 697/1412] 10.222.0.30 () {54 vars in 918 bytes} [Tue Jul 12 14:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:32.822192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513f2d-71b9-41fc-a992-5ff243386326 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.822482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513f2d-71b9-41fc-a992-5ff243386326 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.822759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd513f2d-71b9-41fc-a992-5ff243386326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.892105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:32.895640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-bd513f2d-71b9-41fc-a992-5ff243386326 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:32.896867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bd513f2d-71b9-41fc-a992-5ff243386326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:33.011327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 716/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Jul 12 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:35.512316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.512860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.512976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.514333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:35.520149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d847-86f4-4d37-8063-97a9685a4c4e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.520565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d847-86f4-4d37-8063-97a9685a4c4e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.520697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d847-86f4-4d37-8063-97a9685a4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.585586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7058d847-86f4-4d37-8063-97a9685a4c4e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:35.586779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7058d847-86f4-4d37-8063-97a9685a4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.640038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:35.681602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:35.725364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:35.726886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.870490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872541d1-bff1-4428-ac2a-6903f3b3fc80 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:35.941278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:35.980793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:35.981397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.981680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.981956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.983582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.071061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.072650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbb34b-1b99-4b24-8887-9b3166d7f20c service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.090806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 699/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:36.980921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.981541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.981541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.982882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:36.988473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257aa1-01af-4f0a-856f-eb8b151b24ac None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.988981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257aa1-01af-4f0a-856f-eb8b151b24ac None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.989086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257aa1-01af-4f0a-856f-eb8b151b24ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.062892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8d257aa1-01af-4f0a-856f-eb8b151b24ac None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:37.064736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8d257aa1-01af-4f0a-856f-eb8b151b24ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.090673 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:37.160044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:37.311697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.313133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.457228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615866d2-0c7f-430d-949f-34b64782cc00 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.496968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:37.577399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.577972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.578258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.578537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.580740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.668735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.670359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69760649-01f3-4046-ab80-29da3463cab7 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.687879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 719/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:38.570674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.571217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.571217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.572548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:38.577587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebee31-74f0-42c6-a79e-a171a3773ca3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.578001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebee31-74f0-42c6-a79e-a171a3773ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.578149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebee31-74f0-42c6-a79e-a171a3773ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.631110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:38.653490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f6ebee31-74f0-42c6-a79e-a171a3773ca3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:38.654714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f6ebee31-74f0-42c6-a79e-a171a3773ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.749547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:38.900478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:38.901973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.044243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169786d7-66e0-4eec-ba38-4959d426f953 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.056597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:39.156626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:39.157213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.158038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.158317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.159938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.245690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.247265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a552d92-f7b7-414a-9db2-8f95014219ef service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.264859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 702/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:40.121260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.121841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.121841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.123206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:40.128806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585cd6c-1e22-4ffb-b5ac-b411186e9202 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.129257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585cd6c-1e22-4ffb-b5ac-b411186e9202 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.129404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585cd6c-1e22-4ffb-b5ac-b411186e9202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.147711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:40.203096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1585cd6c-1e22-4ffb-b5ac-b411186e9202 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:40.204319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1585cd6c-1e22-4ffb-b5ac-b411186e9202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.301593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:40.450326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.451840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.559662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:40.594693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006ca5f-bdf8-46c4-8f2c-7b581248ea34 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.706163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.706735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.707026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.707356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.709598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.785348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:40.798317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.799985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437f83e-c833-4d5a-9b35-452e6a1031fb service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.818061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 722/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:41.675224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.675819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.675819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.677138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:41.682146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad39ea-1c52-4dfc-9efc-542bbdaa1ab2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.682573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad39ea-1c52-4dfc-9efc-542bbdaa1ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.682702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad39ea-1c52-4dfc-9efc-542bbdaa1ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.756722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-90ad39ea-1c52-4dfc-9efc-542bbdaa1ab2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:41.757872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90ad39ea-1c52-4dfc-9efc-542bbdaa1ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:41.853355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:42.003380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.005214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:42.088136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:42.150466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cad0dd-efac-4a59-b3f1-b9c8fecc2ac6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.267735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.268392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.268678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.268954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.270656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:42.325865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:42.369710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.371204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3f075-25ce-4185-bed5-5d4c5083a0bc service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.392361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 705/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:43.264307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.264840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.264948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.266303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:43.271427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f8d7-fef0-4722-9b9e-e55861d80c53 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.271845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f8d7-fef0-4722-9b9e-e55861d80c53 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.272909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f8d7-fef0-4722-9b9e-e55861d80c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.347520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a299f8d7-fef0-4722-9b9e-e55861d80c53 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:43.348732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a299f8d7-fef0-4722-9b9e-e55861d80c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.446526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:43.598828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:43.600301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.648886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:43.751269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc02f1-27a2-4455-8b92-08ff32362961 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:43.862157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:43.862732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.863022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.863303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.864950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.871767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:43.956550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:43.958201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87defea9-2dfd-4fd6-8e04-f715ad40c7d8 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:43.976109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 725/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:44.895058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.895718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.895718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.896999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:44.901988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8d66-2d45-497f-aaba-d72986acf513 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.902414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8d66-2d45-497f-aaba-d72986acf513 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.902544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8d66-2d45-497f-aaba-d72986acf513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.979122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3a8a8d66-2d45-497f-aaba-d72986acf513 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:44.980324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3a8a8d66-2d45-497f-aaba-d72986acf513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.048706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:45.076061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:45.227101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:45.228616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.245979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:45.372719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3e341-49a9-4d84-92ce-afeb5002a924 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:45.458891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:45.484156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:45.484742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.485022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.485350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.486972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.576454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:45.578078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb497-5dca-4c90-8ded-02eb749b3c06 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:45.596481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 708/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:46.457866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.458399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.458502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.459814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:46.464607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576231-ba97-4366-9140-18c24311582c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.465004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576231-ba97-4366-9140-18c24311582c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.465182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576231-ba97-4366-9140-18c24311582c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.537536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e3576231-ba97-4366-9140-18c24311582c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:46.538647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e3576231-ba97-4366-9140-18c24311582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.578531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:46.634633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:46.783261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:46.784770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.925978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae782e-b230-4732-b7fe-34c06ed0bd0f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:46.978989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:47.037250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:47.037895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:47.038182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:47.038461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.040102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.126755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:47.128348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ff0f9-4972-4ced-ae23-e33d4f60b22e service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:47.146223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 728/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:48.028571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:48.029066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:48.029204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:48.030702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:48.035756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3073129-0e6b-4f98-8b5d-56ada6bdcf6a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:48.036161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3073129-0e6b-4f98-8b5d-56ada6bdcf6a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:48.036328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3073129-0e6b-4f98-8b5d-56ada6bdcf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:48.113749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a3073129-0e6b-4f98-8b5d-56ada6bdcf6a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:48.114982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a3073129-0e6b-4f98-8b5d-56ada6bdcf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:48.117249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:48.212823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:48.363067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:48.364622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:48.509253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985b99-33be-439c-9f92-3cfb296c7c69 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:48.535975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:48.624145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:48.624803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:48.625125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:48.625447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:48.627089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:48.716120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:48.717775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39f12-0246-4236-a90a-6d455ec47e88 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:48.735336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 711/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:50.657821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.658354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.658354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.659689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:50.664945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be31ba-1fd9-4162-89f6-4abe76325a15 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.665407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be31ba-1fd9-4162-89f6-4abe76325a15 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.665552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be31ba-1fd9-4162-89f6-4abe76325a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.705051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:50.729316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e9be31ba-1fd9-4162-89f6-4abe76325a15 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:50.730505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e9be31ba-1fd9-4162-89f6-4abe76325a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:50.831770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:50.869123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.870685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:50.972958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dbbc7-45e2-44b1-bfb3-6e96e5ed381e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:51.004910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:51.045376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:51.046021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.046306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.046583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.048997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.155780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:51.157269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59518e3e-56e8-40ba-aab9-141fab8fa246 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:51.175837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 731/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:53.439451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.440053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.440053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.441488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:53.446906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece4e0-61c1-428b-820b-96e2e3f6fec2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.447339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece4e0-61c1-428b-820b-96e2e3f6fec2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.447482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece4e0-61c1-428b-820b-96e2e3f6fec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.512200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-61ece4e0-61c1-428b-820b-96e2e3f6fec2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:53.513442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-61ece4e0-61c1-428b-820b-96e2e3f6fec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.526514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.610123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:53.648171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.649760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.753052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225d4e-3fb0-4be5-b505-420d18f7137a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.765055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.825216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.825857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.826137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.826411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.828026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.940422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.942022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff5126-6dd9-45af-9f26-39030b6ecb22 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.961282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 714/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:56.326951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.327503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.327503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.329568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:56.334898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b09f-5f5f-483b-b9d3-ec6657d4d448 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.335338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b09f-5f5f-483b-b9d3-ec6657d4d448 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.335465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b09f-5f5f-483b-b9d3-ec6657d4d448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.382087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:56.399661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9d72b09f-5f5f-483b-b9d3-ec6657d4d448 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:56.400965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9d72b09f-5f5f-483b-b9d3-ec6657d4d448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:56.495478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:56.533908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.535474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:56.649699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed9506-bb1b-4bdf-b7b1-4cbc1b9a7e41 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.696062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:56.729856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.730522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.730793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.731073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.732731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:56.840187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.841586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0d9ef-d071-4bbc-9c27-68a4a518b653 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.859876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 734/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:59.009112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.009962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.009962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.011325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:59.016593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2aee7-1358-49d1-bb90-88ad3d2d2714 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.017015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2aee7-1358-49d1-bb90-88ad3d2d2714 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.017151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2aee7-1358-49d1-bb90-88ad3d2d2714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.079864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4ad2aee7-1358-49d1-bb90-88ad3d2d2714 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:59.081063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4ad2aee7-1358-49d1-bb90-88ad3d2d2714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.101303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:59.179664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:59.217275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:59.218783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.321830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0a070-2ae3-4be0-82b2-148d18a69a16 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:59.341377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:59.395064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:59.395701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.395983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.396256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.397915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.505398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:59.506838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a9b96-1346-415e-806c-c8d34360de25 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:59.525328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 717/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:01.688780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:01.689362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:01.689362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:01.690780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:01.700989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7923-688e-44ce-99d6-62b94dbbe59e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:01.700989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7923-688e-44ce-99d6-62b94dbbe59e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:01.700989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e7923-688e-44ce-99d6-62b94dbbe59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:01.758160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:01.761280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-550e7923-688e-44ce-99d6-62b94dbbe59e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:01.762487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-550e7923-688e-44ce-99d6-62b94dbbe59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:01.858378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:01.896024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:01.897696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:01.997858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c919-2940-46fb-b3d4-f92d569769c7 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.045659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:02.070155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.070811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.071101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.071382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.073081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.179292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.180719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7fc7b-b784-4cc5-9672-a31bf93f9787 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.199251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 737/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:04.391357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.391933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.391933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.393380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:04.398617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d615-fbb0-4189-8182-3b536c729736 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.399058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d615-fbb0-4189-8182-3b536c729736 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.399169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d615-fbb0-4189-8182-3b536c729736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.463894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0744d615-fbb0-4189-8182-3b536c729736 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:04.465099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0744d615-fbb0-4189-8182-3b536c729736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.494370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.560631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:04.598855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.600411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.704256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1d50d-e37a-4111-93ff-cf6809897f80 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.731755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.777249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.777899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.778186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.778543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.780138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.888873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.890272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552de7c-2605-4be0-8c01-29544a37d1cf service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.907984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 720/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:06.970673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.971243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.971243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.972614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:06.977862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1582fd-f251-43be-8310-4c2261785678 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.978317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1582fd-f251-43be-8310-4c2261785678 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.978425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1582fd-f251-43be-8310-4c2261785678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.042886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2a1582fd-f251-43be-8310-4c2261785678 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:07.044100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2a1582fd-f251-43be-8310-4c2261785678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.053024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:07.141473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:07.178361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.179966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.282997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296efa22-76c9-4a1b-984c-831e51ae1370 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.336849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:07.355435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.356571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.356854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.357129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.359660 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.468086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.470168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2870929-bbf5-4516-b5ac-d73bedcd4e9a service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.489186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 740/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:09.552122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.552683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.552683 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.554148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:09.559549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155464eb-747d-45b9-a1b0-4b7049fd20bd None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.559955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155464eb-747d-45b9-a1b0-4b7049fd20bd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.560117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155464eb-747d-45b9-a1b0-4b7049fd20bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.623421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-155464eb-747d-45b9-a1b0-4b7049fd20bd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:09.625140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-155464eb-747d-45b9-a1b0-4b7049fd20bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.662954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:09.720025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:09.756546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:09.758091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.861573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282d2ce-4755-4a7e-8bfd-3c3479786b50 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:09.899585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:09.935676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:09.936322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.936609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.936887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.938599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:10.045721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:10.047164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f547f-e8dd-4ad9-ab7f-86d0b6ebaca5 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:10.066280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 723/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:12.169076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.169668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.169668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.171051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:12.176450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c1f4-a8e9-49d4-820b-853b334b894c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.176854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c1f4-a8e9-49d4-820b-853b334b894c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.177004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4c1f4-a8e9-49d4-820b-853b334b894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.239628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-90d4c1f4-a8e9-49d4-820b-853b334b894c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:12.240839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-90d4c1f4-a8e9-49d4-820b-853b334b894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:12.258164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:12.336062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:12.381898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:12.383462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:12.606240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419b94e-b9e2-4b6e-b6e4-52709c3413c4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:12.672955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:12.679515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:12.680170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.680454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.680730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.682406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:12.789717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:12.791067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0db5ec-10f0-4f7b-bb08-8b8845528281 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:12.808497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 743/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:14.844194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.844717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.844717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.846095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:14.851413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb41dc5-7174-4177-a871-1cc8dcfee298 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.851788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb41dc5-7174-4177-a871-1cc8dcfee298 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.851994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb41dc5-7174-4177-a871-1cc8dcfee298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.915372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7cb41dc5-7174-4177-a871-1cc8dcfee298 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:14.916577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7cb41dc5-7174-4177-a871-1cc8dcfee298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:14.964059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:15.012537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:15.048662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.050213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.152209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1aa85-9f10-413b-84d2-cbafacc2caba None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.198885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:15.224053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.224899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.225398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.225965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.227658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.336682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.339425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20860f6-40a6-4a6b-9458-89dd8d726105 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.361942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 726/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:25.010735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=22 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:25.359040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=66 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.036443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1a7b-c193-4ee8-a18b-71f2ff2e7e61 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.037009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1a7b-c193-4ee8-a18b-71f2ff2e7e61 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.037009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1a7b-c193-4ee8-a18b-71f2ff2e7e61 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.038418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 745/1471] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:35.043834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751984f-cf56-406a-9135-54d81e2b3535 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.044268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751984f-cf56-406a-9135-54d81e2b3535 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.044391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751984f-cf56-406a-9135-54d81e2b3535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.108764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c751984f-cf56-406a-9135-54d81e2b3535 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:35.109984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c751984f-cf56-406a-9135-54d81e2b3535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.203273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 727/1472] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:35.286332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.286670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.286902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.288333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 746/1473] 10.222.0.30 () {54 vars in 962 bytes} [Tue Jul 12 14:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:35.293043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b4fe-12fc-43de-bde6-0b9b1818ce65 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.293367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b4fe-12fc-43de-bde6-0b9b1818ce65 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.293653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b4fe-12fc-43de-bde6-0b9b1818ce65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.367337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4604b4fe-12fc-43de-bde6-0b9b1818ce65 None None] MFA Rules not processed for user `401f40d749db4643824d38db0cf533cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:35.368539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4604b4fe-12fc-43de-bde6-0b9b1818ce65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.389558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.446778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 728/1474] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:35.454836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.456349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.558404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954bb6a-7741-4830-b857-fe904951bd7d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.628250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.628885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.629185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.629458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.631078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.683681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.738332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.739800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9ab17-b2d2-47d4-964f-5bbe5dc9cbcf service glance] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.758807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 747/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:37.875862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:37.876397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:37.876397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:37.877757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 729/1476] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:33:37.883015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be8aa-39cc-4ef1-8c4d-615570cb5229 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:37.883425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be8aa-39cc-4ef1-8c4d-615570cb5229 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:37.883583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be8aa-39cc-4ef1-8c4d-615570cb5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:37.947854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cc6be8aa-39cc-4ef1-8c4d-615570cb5229 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:37.949058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cc6be8aa-39cc-4ef1-8c4d-615570cb5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:38.045623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 748/1477] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:38.070197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:38.071783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:38.194902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89f49-5933-4ccc-8710-653adbc183e8 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:38.242919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:38.289634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:38.290297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:38.290583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:38.290861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:38.291593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:38.293421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:33:38.293421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:33:38.293421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:33:38.294612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ca04e-dc15-40ee-bd6a-8100d8c9fa9d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:38.314485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 730/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Jul 12 14:33:38 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:40.467149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:40.467630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:40.467737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:40.469089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 749/1479] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:40.474233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875f35-74d6-48cb-b945-ee8865b632ca None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:40.474622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875f35-74d6-48cb-b945-ee8865b632ca None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:40.474791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875f35-74d6-48cb-b945-ee8865b632ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:40.537939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-36875f35-74d6-48cb-b945-ee8865b632ca None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:40.539082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-36875f35-74d6-48cb-b945-ee8865b632ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:40.632734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 731/1480] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:40.675777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:40.677391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:40.687056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:40.820132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6134b49-f9ca-4d42-9d40-b65a1dd7f013 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:40.898554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:40.931009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:40.931645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:40.931925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:40.932201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:40.933875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:41.020845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:41.022450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684914c-8393-40b6-8638-a54b81fce797 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:41.039907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 750/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:43.260904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.261468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.261574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.263106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 732/1482] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:43.269177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a49e3-1809-496a-9353-6445b0b50aa5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.269609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a49e3-1809-496a-9353-6445b0b50aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.269763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a49e3-1809-496a-9353-6445b0b50aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.334574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-348a49e3-1809-496a-9353-6445b0b50aa5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:43.335778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-348a49e3-1809-496a-9353-6445b0b50aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.381908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:43.432829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 751/1483] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:43.469115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:43.470682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.495513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:43.576511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69151-8268-4621-8ce6-e19ab3173d61 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:43.648577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:43.649249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.649531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.649808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.651458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.724727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:43.760471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:43.761873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d617bba-5936-43ee-b09e-46a758b8831f service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:43.782809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 733/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:45.862473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.862986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.863083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.864432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 752/1485] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:45.869777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4fb9-2286-4ec9-a683-6d6acc357859 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.870206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4fb9-2286-4ec9-a683-6d6acc357859 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.870347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4fb9-2286-4ec9-a683-6d6acc357859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.933900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7a6f4fb9-2286-4ec9-a683-6d6acc357859 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:45.935767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7a6f4fb9-2286-4ec9-a683-6d6acc357859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:46.029391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 734/1486] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:46.067920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:46.069493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:46.178371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df95ce1-4e56-4842-a73b-a3b4fd8e94a3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:46.249680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:46.251406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:46.252046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:46.252328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:46.252601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:46.254281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:46.362006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:46.363364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede167b-5f58-4cc5-81ab-6ee09e157373 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:46.380684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 753/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:48.538118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.538650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.538650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.539961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 735/1488] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:48.545209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd32-7401-4fd7-8f35-23bdcb6f8074 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.545594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd32-7401-4fd7-8f35-23bdcb6f8074 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.545803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4cd32-7401-4fd7-8f35-23bdcb6f8074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.612270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dab4cd32-7401-4fd7-8f35-23bdcb6f8074 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:48.613518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dab4cd32-7401-4fd7-8f35-23bdcb6f8074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.666381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:48.709166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 754/1489] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:48.749310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.750846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.786292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:48.853954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983292-6fe3-4684-b3f6-4fff4474eb0d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.928789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.929496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.929780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.930055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.931701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.011258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:49.040833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.042379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e8706-b516-4cb4-936c-31d08acb2d1c service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.061229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 736/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:51.523872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.524423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.524423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.525851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 755/1491] 10.222.0.30 () {54 vars in 926 bytes} [Tue Jul 12 14:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:51.531591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141233-6647-4bb5-967f-e980115a3129 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.531999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141233-6647-4bb5-967f-e980115a3129 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.532162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141233-6647-4bb5-967f-e980115a3129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.598476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e9141233-6647-4bb5-967f-e980115a3129 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:51.599701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e9141233-6647-4bb5-967f-e980115a3129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.695472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 737/1492] 10.222.0.30 () {58 vars in 1023 bytes} [Tue Jul 12 14:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:51.733709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:51.735360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.835808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c58a6-5eba-4d50-bdb3-ed4b9a77ab3f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:51.907975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:51.908612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.908893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.909186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.910823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.919784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:52.019368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:52.020720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9da13-97c4-49a9-8d13-0fe9c0fb25b4 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:52.038844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 756/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:01.693699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=82 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:02.036610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=50 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:32.026534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:32.027131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:32.027131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:32.100392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:32.102279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:32.197241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 738/1494] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:32.205331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:32.205832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:32.205944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:32.279923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:32.281120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:32.378314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 757/1495] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:32.383729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:32.385003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:32.396552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb8e60-093f-40d2-b28a-5b8834841dc1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:32.456978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:32.490433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:32.491102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:32.491399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:32.491681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:32.492415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:35:32.492728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:32.493786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:32.493786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:35:32.493786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:35:32.494691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:32.506017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 739/1496] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:35:32.515035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:32.526573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfc529-5f31-45f0-a116-ce006082f42e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:32.588934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:32.621273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:32.621932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:32.622213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:32.622485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:32.623186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:32.624917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:32.624917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:35:32.624917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:35:32.626137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:32.734099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 758/1497] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:35:32 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:35:32.738281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:32.739808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:32.772271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a0de-dcc9-49ed-bf17-4295a46a36c7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:32.866817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:32.867392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:32.867670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:32.867939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:32.868628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:32.869667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:32.869667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:35:32.869667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:35:32.870880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:32.979801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 740/1498] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:35:32 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:35:32.983920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:32.985478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.046214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3329d-f31d-411b-9e98-e471d0d823f9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:33.128935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:33.139497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:33.141018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.141018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.141018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.141417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:33.142417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:33.142417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:35:33.142417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:35:33.143288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:33.155916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 759/1499] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:35:33.159892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:33.161527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.173044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356c83f-9919-4865-8a90-46a2f1fadf83 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:33.199819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:33.270199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:33.270862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.271147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.271423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.272147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:33.273204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:33.273204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:35:33.273204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:35:33.274086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fee4-2331-4ae3-9f81-26b058a077f0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:33.286004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 741/1500] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:35:33.290914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:33.292498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.325945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b23cd-326e-4714-859c-03aba1c31be8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:33.417799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:33.418605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.418888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.419163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] PATH_INFO: `/v3/projects/e04e897ed7fc45bca23ef28c4192a2c3/users/0bb80f3af3bc4d8b9c0767ddfaf3e42f/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.441220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04e897ed7fc45bca23ef28c4192a2c3, user_id=0bb80f3af3bc4d8b9c0767ddfaf3e42f, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:33.442303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:33.442303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:35:33.442303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:35:33.444198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ffcd1-1739-46cf-b8c3-fa845f6cc4c5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:33.504252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 760/1501] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:35:33 2022] PUT /identity/v3/projects/e04e897ed7fc45bca23ef28c4192a2c3/users/0bb80f3af3bc4d8b9c0767ddfaf3e42f/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:35:33.516846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42216ca-d77f-4219-9edc-6d09c22cb0bc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.517140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42216ca-d77f-4219-9edc-6d09c22cb0bc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.517453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42216ca-d77f-4219-9edc-6d09c22cb0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.571904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f42216ca-d77f-4219-9edc-6d09c22cb0bc None None] MFA Rules not processed for user `0bb80f3af3bc4d8b9c0767ddfaf3e42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:33.573092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f42216ca-d77f-4219-9edc-6d09c22cb0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.649903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 742/1502] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:33.662731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.663027 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.663309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.665296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 761/1503] 10.222.0.30 () {54 vars in 963 bytes} [Tue Jul 12 14:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:35:33.669953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70500c-fd01-4b34-a1b8-9672e3ba6401 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.670188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70500c-fd01-4b34-a1b8-9672e3ba6401 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.670471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70500c-fd01-4b34-a1b8-9672e3ba6401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.699354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:33.743100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bc70500c-fd01-4b34-a1b8-9672e3ba6401 None None] MFA Rules not processed for user `177ef85784a94f3782aa48b60a6fb8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:33.744304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bc70500c-fd01-4b34-a1b8-9672e3ba6401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.818127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 743/1504] 10.222.0.30 () {58 vars in 1060 bytes} [Tue Jul 12 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:33.826311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:33.827820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.876812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:33.930715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c198d-be58-4cbf-8500-b375b5bd5205 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:34.003014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:34.003654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.003935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.004210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.005887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:34.103934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:34.105407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:34.108878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:34.122875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 762/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:34.165369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2e51e-44c9-4742-8323-a9270f0d23f2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.165783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2e51e-44c9-4742-8323-a9270f0d23f2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.165932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2e51e-44c9-4742-8323-a9270f0d23f2 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.167217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 744/1506] 10.222.0.30 () {52 vars in 829 bytes} [Tue Jul 12 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:35:34.170778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:34.171998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:34.276731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acbdc0-1326-4966-8b56-257151f5ea8d service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:34.368579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:34.369477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.369760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.370034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] PATH_INFO: `/v3/users/0bb80f3af3bc4d8b9c0767ddfaf3e42f` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.382303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb80f3af3bc4d8b9c0767ddfaf3e42f)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:34.383374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:34.383374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:35:34.383374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:35:34.384582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:34.394068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:34.567521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb80f3af3bc4d8b9c0767ddfaf3e42f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:34.567990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb80f3af3bc4d8b9c0767ddfaf3e42f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:34.568418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb80f3af3bc4d8b9c0767ddfaf3e42f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:34.581233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb80f3af3bc4d8b9c0767ddfaf3e42f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:34.599912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb80f3af3bc4d8b9c0767ddfaf3e42f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:34.687167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 763/1507] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:35:34 2022] DELETE /identity/v3/users/0bb80f3af3bc4d8b9c0767ddfaf3e42f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:35:34.691508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.691798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.692077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.866465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:34.867610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:34.942037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:34.954414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 745/1508] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:34.963965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:34.965219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:35.063640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9019-0fc6-4b2f-8bf1-c9567736672a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:35.119670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:35.137901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:35.138570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:35.138856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:35.139134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:35.140771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:35.257079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:35.258584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ed360-bad3-4a97-a6f9-d36e53b0f443 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:35.277445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 764/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:35.740519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:35.742342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:35.753990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a9a3b-291b-43a9-b2d0-53ec9d5a1784 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:35.852196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:35.853044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:35.853369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:35.853626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] PATH_INFO: `/v3/projects/e04e897ed7fc45bca23ef28c4192a2c3` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:35.863742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04e897ed7fc45bca23ef28c4192a2c3)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:35.864763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:35:35.864763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:35:35.864763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:35:35.865979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:35.977149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04e897ed7fc45bca23ef28c4192a2c3', 'request_id': 'req-eb6517a7-4511-4920-a6c2-3a690626a5e3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:35.983356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04e897ed7fc45bca23ef28c4192a2c3', 'request_id': 'req-eb6517a7-4511-4920-a6c2-3a690626a5e3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:36.058554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04e897ed7fc45bca23ef28c4192a2c3', 'request_id': 'req-eb6517a7-4511-4920-a6c2-3a690626a5e3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:36.090397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] The token cache is being invalidate because project e04e897ed7fc45bca23ef28c4192a2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:35:36.090686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:36.091205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb6517a7-4511-4920-a6c2-3a690626a5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:36.092538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 746/1510] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:35:35 2022] DELETE /identity/v3/projects/e04e897ed7fc45bca23ef28c4192a2c3 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:35:45.275210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=62 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:46.088369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:17.709856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:17.710446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:17.710446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:17.711721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:37:17.716523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2850b6-caf8-42a7-9106-bd8146f8b325 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:17.716934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2850b6-caf8-42a7-9106-bd8146f8b325 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:17.717087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2850b6-caf8-42a7-9106-bd8146f8b325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:17.795868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6e2850b6-caf8-42a7-9106-bd8146f8b325 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:17.797055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6e2850b6-caf8-42a7-9106-bd8146f8b325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:17.903916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:18.060461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:18.062585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:18.185063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0603-eb74-4c0c-bade-57f278cbde05 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:18.191338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:18.194226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:18.194226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:18.194226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:18.200878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:18.202531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385277d7-48d0-4382-a23a-9bccb47484ab service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:18.212271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 766/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:19.036662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727e8f-4919-46bc-bde9-44cbb55b5ed0 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:19.037278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727e8f-4919-46bc-bde9-44cbb55b5ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:19.037278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727e8f-4919-46bc-bde9-44cbb55b5ed0 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:19.038610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:19.043580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b48a-2e1e-4ea5-97f5-c40ee330a5fe None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:19.043979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b48a-2e1e-4ea5-97f5-c40ee330a5fe None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:19.044144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b48a-2e1e-4ea5-97f5-c40ee330a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:19.061824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f049b48a-2e1e-4ea5-97f5-c40ee330a5fe None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:19.063016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f049b48a-2e1e-4ea5-97f5-c40ee330a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:19.071824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:19.227315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d727e8f-4919-46bc-bde9-44cbb55b5ed0 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:19.241994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d727e8f-4919-46bc-bde9-44cbb55b5ed0 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:19.247362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:19.247988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:19.248272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:19.248548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:19.257415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:19.258990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e24f3-d86e-40e0-9c49-679561b673e6 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:19.268260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 749/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:20.248150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6cd-132f-4f97-860f-254e9d607fac None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:20.248716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6cd-132f-4f97-860f-254e9d607fac None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:20.248716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6cd-132f-4f97-860f-254e9d607fac None None] PATH_INFO: `/v3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:20.250091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 768/1517] 10.222.0.33 () {54 vars in 925 bytes} [Tue Jul 12 14:37:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:37:20.256569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f771-7c95-45ea-a7c0-511dc1adab6c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:20.257007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f771-7c95-45ea-a7c0-511dc1adab6c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:20.257138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f771-7c95-45ea-a7c0-511dc1adab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:20.314216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8552f771-7c95-45ea-a7c0-511dc1adab6c None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:20.315426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8552f771-7c95-45ea-a7c0-511dc1adab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:20.331690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:20.386650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 750/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Jul 12 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:29.057148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=56 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:30.296445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf16e-d99d-414e-8e75-88d04ec6f30d None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.297075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf16e-d99d-414e-8e75-88d04ec6f30d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.297075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf16e-d99d-414e-8e75-88d04ec6f30d None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.298917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 769/1519] 10.222.0.30 () {54 vars in 960 bytes} [Tue Jul 12 14:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:30.303800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8712d8d-39bd-44af-87df-093b1068eafb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.304232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8712d8d-39bd-44af-87df-093b1068eafb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.304374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8712d8d-39bd-44af-87df-093b1068eafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.322200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d8712d8d-39bd-44af-87df-093b1068eafb None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:30.323399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d8712d8d-39bd-44af-87df-093b1068eafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:30.332248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 751/1520] 10.222.0.30 () {58 vars in 1057 bytes} [Tue Jul 12 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:30.339574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf16e-d99d-414e-8e75-88d04ec6f30d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:30.353605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf16e-d99d-414e-8e75-88d04ec6f30d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:30.358853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:30.359496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.359785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.360064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.369215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:30.370803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8f7fe-2846-4de1-a9cc-642a30eef5ab service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:30.380847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 770/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:30.526312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41457-9d0b-4057-8a86-0aa697194783 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.526601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41457-9d0b-4057-8a86-0aa697194783 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.526879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c41457-9d0b-4057-8a86-0aa697194783 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.528291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:30.533704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849381e2-dfec-41d4-bed5-0f595a3ed96a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.533989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849381e2-dfec-41d4-bed5-0f595a3ed96a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.534273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849381e2-dfec-41d4-bed5-0f595a3ed96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.552386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-849381e2-dfec-41d4-bed5-0f595a3ed96a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:30.553544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-849381e2-dfec-41d4-bed5-0f595a3ed96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:30.561394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:30.715888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c41457-9d0b-4057-8a86-0aa697194783 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:30.730834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c41457-9d0b-4057-8a86-0aa697194783 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:30.736358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:30.736988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:30.737307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:30.737581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:30.745736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:30.747303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669edcf1-8b87-4dee-8175-8e4cc0d04cad service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:30.757746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 753/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:31.583090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe010c2-d1d0-4323-b0a8-4a4345bf1da8 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:31.583090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe010c2-d1d0-4323-b0a8-4a4345bf1da8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:31.583090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe010c2-d1d0-4323-b0a8-4a4345bf1da8 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:31.583884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:31.588453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceeb1f-0218-4412-8932-8258e9e85a98 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:31.588854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceeb1f-0218-4412-8932-8258e9e85a98 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:31.589009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceeb1f-0218-4412-8932-8258e9e85a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:31.607465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-74ceeb1f-0218-4412-8932-8258e9e85a98 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:31.612864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-74ceeb1f-0218-4412-8932-8258e9e85a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:31.617185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:31.769827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe010c2-d1d0-4323-b0a8-4a4345bf1da8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:31.784450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe010c2-d1d0-4323-b0a8-4a4345bf1da8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:31.790231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:31.790864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:31.791150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:31.791432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:31.800090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:31.801701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260028d-ca8d-4a8a-a687-1fd826381275 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:31.811309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 773/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:35.635482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:35.636047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:35.636047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:35.637466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:35.643236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8555b-0c5d-4dc9-88c0-866d6bfb1ffd None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:35.643633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8555b-0c5d-4dc9-88c0-866d6bfb1ffd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:35.643829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8555b-0c5d-4dc9-88c0-866d6bfb1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:35.662511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b1f8555b-0c5d-4dc9-88c0-866d6bfb1ffd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:35.663745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b1f8555b-0c5d-4dc9-88c0-866d6bfb1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:35.671755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:35.743722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:35.745464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:35.823690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290295eb-7f4a-45ff-ac16-1167ccea9a1e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:35.827223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:35.827855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:35.828135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:35.828409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:35.837662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:35.839079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc0435-0efe-4e2b-a68d-8380d78e2536 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:35.848883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 756/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:37.896389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8859c47-b97c-45b0-9237-a23f543cf059 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:37.896918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8859c47-b97c-45b0-9237-a23f543cf059 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:37.896918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8859c47-b97c-45b0-9237-a23f543cf059 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:37.898321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:37.903427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86baa-16fd-4072-ab37-818f6de72066 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:37.903833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86baa-16fd-4072-ab37-818f6de72066 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:37.903998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86baa-16fd-4072-ab37-818f6de72066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:37.921226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a4a86baa-16fd-4072-ab37-818f6de72066 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:37.922951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a4a86baa-16fd-4072-ab37-818f6de72066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:37.930866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:37.966610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8859c47-b97c-45b0-9237-a23f543cf059 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:37.978392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8859c47-b97c-45b0-9237-a23f543cf059 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:37.981994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:37.982633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:37.982921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:37.983202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:37.992683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:37.994144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a923c-8853-4b08-bb1e-f1e6dad56eca service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:38.003978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 776/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:41.439208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:41.439765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:41.439765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:41.441129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:41.446416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683da05-08d3-4bb6-a4cd-701e259c9f52 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:41.446860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683da05-08d3-4bb6-a4cd-701e259c9f52 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:41.446967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683da05-08d3-4bb6-a4cd-701e259c9f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:41.465529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4683da05-08d3-4bb6-a4cd-701e259c9f52 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:41.469869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4683da05-08d3-4bb6-a4cd-701e259c9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:41.474594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:37:41.778291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:41.779991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:41.845033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:41.858013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa22b2-71a4-49b5-b14e-d8e4ca9077b9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:41.861612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:41.862260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:41.862541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:41.862818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:41.871816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:41.873340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d5453-be64-446d-8ec0-f98cfd2fca9d service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:41.883531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 759/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:43.941304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:43.941886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:43.941886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:43.943250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:37:43.948530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b65e7-3ed4-4e84-8dd2-f320b19bd06a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:43.948933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b65e7-3ed4-4e84-8dd2-f320b19bd06a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:43.949088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b65e7-3ed4-4e84-8dd2-f320b19bd06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:43.967274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7c2b65e7-3ed4-4e84-8dd2-f320b19bd06a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:43.968481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7c2b65e7-3ed4-4e84-8dd2-f320b19bd06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:43.977063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:37:44.284911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:44.287291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:44.299411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd13bf0-2db3-406b-9724-f5ce3030bafa None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:44.302993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:44.303607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:44.303893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:44.304168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:44.313813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:44.315237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459151b-bc16-4415-b94d-a251b1b7aad0 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:44.324533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 779/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:53.962387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=11 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:54.320467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=33 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:12.009113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e361-05be-475a-ac72-fcbf2c01a00a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.012075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e361-05be-475a-ac72-fcbf2c01a00a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.012075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964e361-05be-475a-ac72-fcbf2c01a00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.034690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e964e361-05be-475a-ac72-fcbf2c01a00a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.036379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e964e361-05be-475a-ac72-fcbf2c01a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.045137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 761/1540] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.053710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.054681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.055179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.078382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.079972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.087985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d490-2011-48b6-9f21-743c410ec742 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.088344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d490-2011-48b6-9f21-743c410ec742 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.088550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d490-2011-48b6-9f21-743c410ec742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.099198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 780/1541] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.104413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.108211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-fc67d490-2011-48b6-9f21-743c410ec742 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.109619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fc67d490-2011-48b6-9f21-743c410ec742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.119347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f1bb-eab4-41bb-9ac9-7e0ed07ad274 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.122002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 762/1542] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:12.127838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.128809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.129338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.129857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.131735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.132262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.132755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.137623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:12.139412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.141263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:12.141263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:39:12.141263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:39:12.146013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9d30-d250-42e3-8246-075190b5fcca admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.165802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.168577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.172351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 781/1543] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:39:12.173925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.175442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.176359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.189592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 763/1544] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:12.191555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.211858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.213135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a1d6-ed35-4930-9216-ce98bce2e423 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.214588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.222808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.224270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.225742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.227482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.228544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.229926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:12.229926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:39:12.229926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:39:12.231596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af85d8-34de-4082-9314-bd648aa223a3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.235647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 782/1545] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.236330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.252577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d974dc0-e99c-4713-9f06-59c3d8bf28bf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.257974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.258890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.259415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.259981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.260883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:12.261384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.263202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81417a-276f-45e3-afe8-9097d5c5125c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.288680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 783/1546] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:12.290539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.291041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.291522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.321845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.323132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.332122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 784/1547] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.332981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.347063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5aa3c-6b4d-4f3d-b231-2cc765c8e457 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.348128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 764/1548] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:12.352931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b74dc6-ac47-4258-b579-5c71ec58cb86 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.353859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b74dc6-ac47-4258-b579-5c71ec58cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.354152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b74dc6-ac47-4258-b579-5c71ec58cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.378373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e6b74dc6-ac47-4258-b579-5c71ec58cb86 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.379602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e6b74dc6-ac47-4258-b579-5c71ec58cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.419198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.419976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.420354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.420705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.422052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.423973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.451460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 765/1549] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.452864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.453218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.453508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.474802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.476052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.486960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 766/1550] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.486960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.495611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b449f2-2379-4476-8644-4d396a1862d8 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.503068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.503068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.503068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.503068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.503601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:12.503601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.504694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e9b15-7c30-441a-a2bc-cff9c7571095 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.516699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 767/1551] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:12.517877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa00c4-6fcc-4cd7-82ff-20c1c76461bc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.518169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa00c4-6fcc-4cd7-82ff-20c1c76461bc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.518451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa00c4-6fcc-4cd7-82ff-20c1c76461bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.531867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 785/1552] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/projects => generated 391 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:12.533468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.537662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-05aa00c4-6fcc-4cd7-82ff-20c1c76461bc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.538862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-05aa00c4-6fcc-4cd7-82ff-20c1c76461bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.595081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeb568-93a0-43cd-9ecf-e2549bdc0d02 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.599896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.600073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 768/1553] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.600572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.600948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.601262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.601392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475c05-cfed-454c-8526-44948cf4739f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.601571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475c05-cfed-454c-8526-44948cf4739f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.601847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475c05-cfed-454c-8526-44948cf4739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.601966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.602970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:12.602970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:39:12.602970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:39:12.604228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170637a2-8722-4d27-a2b8-6aee664b5a46 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.623781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-eb475c05-cfed-454c-8526-44948cf4739f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.625725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-eb475c05-cfed-454c-8526-44948cf4739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.633760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 769/1554] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.635067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa904-f133-46c8-af74-fd8a2bd2ba84 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.635350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa904-f133-46c8-af74-fd8a2bd2ba84 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.635626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaa904-f133-46c8-af74-fd8a2bd2ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.653757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-cabaa904-f133-46c8-af74-fd8a2bd2ba84 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.654909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cabaa904-f133-46c8-af74-fd8a2bd2ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.663214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 770/1555] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.664367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.664649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.664925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.682759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.684155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.691784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 786/1556] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:12.691784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.691784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.691784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.697981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 771/1557] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.697981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.701980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82efd62-bb03-4437-999d-a5cad3795ebd None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.706615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.708377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.708377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.708377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.708645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.710420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.710569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf275-2236-404f-9d51-52b31b011ea4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.711656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.719312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:12.724112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 787/1558] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.724112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.731099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff9b5d-28e2-455d-a912-914acc7626e1 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.736444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.737082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.737565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.737877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.738588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.740543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cbb69-55d5-4f3f-bb31-0c6ebfc12465 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.820232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 788/1559] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:12.821565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.822034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 772/1560] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:12.822170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.823154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.823468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.823718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.825137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.832132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:12.842005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.842573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.843209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.844268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.902602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 789/1561] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.909733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 773/1562] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.909859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.909967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.919030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6204c9-cdc7-4c89-9afb-6a4bc3390c46 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.919220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db60da-e3a4-4c5a-b4a1-a9b311582e8f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.923580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.923741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.924239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.924359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.924525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.924646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.924804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.924921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.925573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:12.925676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.925883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.927214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e6c5b-184f-41c4-bb49-8c215cc71ecd admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.927457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.937698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 774/1563] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:12.938901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.939192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.939468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.943635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 790/1564] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:12.944485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.954756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e15136-3681-41ff-af4c-cdf47ca2d1b5 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.959638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.960288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.960572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.960968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.961706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:12.962025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.963621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.964710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:12.965935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:12.973869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 775/1565] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:12.974632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.990003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 791/1566] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:12.990003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:12.990296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaccb1e-d193-43a6-b768-5e0af5cad492 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.993283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d446-257c-48c0-afc5-189cdd7628dd admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:12.993424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.994032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.994320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.994595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:12.995316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:12.996896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:12.997849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:12.998490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:12.998773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:12.999056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.000284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.002208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.007611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 776/1567] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:13.008386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.018467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284c6f2-9fd2-4a73-990e-a5c666714339 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.023077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.023697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.023976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.024248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.024947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:13.025286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.026849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.039063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 777/1568] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:13.039808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.048975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46ca2-7cfc-4dc3-b775-502b979a155b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.053596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.054202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.054592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.054749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.055445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:13.055746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.057304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.063067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 792/1569] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:12 2022] POST /identity/v3/users => generated 282 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.063067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.076055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 778/1570] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:13.076055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.076345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74e7e-e2bf-47e5-9ce1-a9e7c2f3bcdb admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.076345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.076345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.076630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.076871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.077626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.079729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.080767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1ce7-689f-478d-aab7-b10a1ea307e0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.086022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.086980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.087179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.087471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.088145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.089868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 793/1571] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:13.090075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.090703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.101184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870522-dd95-4523-b7f0-54583e7a43ad admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.106140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.106862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.107150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.107495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.108218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:13.108528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.110263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.122195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 794/1572] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:13.122944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.132428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f72dec-c5d7-4b33-8fc9-5b160ceb2e22 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.137031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.137675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.137960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.138237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.138905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.141331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.176279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 779/1573] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.177108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.236465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5f756-8c6b-453f-b39f-044e5ab1a3b7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.239318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 795/1574] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.242798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.242911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.242911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.242911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.242911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.243287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.244714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.255306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 780/1575] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:13.256095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.267509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29083e-33e0-4577-966a-9da2f1bc43be admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.272466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:13.305177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232335-7395-4b71-a6d7-690ac2cf0d15 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.310130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.310796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.311628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.311910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.312634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.314608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.324580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.325277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.325559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.325834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.326610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.328792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.338735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 781/1576] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:13.340156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.349877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0db991-4bd7-49a3-b823-891d9dc5145e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.355996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.357397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.357397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.357397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.357964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.359867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.369896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:13.377595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 796/1577] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.379517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.436018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 782/1578] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.436018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.446248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46153e0-e290-4717-b03c-629e298c9560 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.499053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fd1b7-1d63-45fc-9407-4486b1457646 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.501527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.502421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.502708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.502988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] PATH_INFO: `/v3/projects/f3eed61b06634aeb92e4008722f02898/users/b4b57128eb75430b950c095737ac1822/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.503924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.504571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.504852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.505135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.505840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.507742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.518927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3eed61b06634aeb92e4008722f02898, user_id=b4b57128eb75430b950c095737ac1822, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.521608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.599044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 783/1579] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.599824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.608238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 797/1580] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:13 2022] PUT /identity/v3/projects/f3eed61b06634aeb92e4008722f02898/users/b4b57128eb75430b950c095737ac1822/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:13.613638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.670078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff856e8b-38ee-4319-a213-f151e3e37c07 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.675442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.676109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.676235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8ff4c-fdbd-436e-bcfd-47988a64eb21 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.676395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.676700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.677414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.679326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.680819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.681507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.681792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.682063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.682741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.684638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.757078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 798/1581] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.758118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.763412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:13.768204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69fa9-8614-4d47-b4b0-47239bebf9a3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.772715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 784/1582] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.772715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.773557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.774429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.774704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.774973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] PATH_INFO: `/v3/projects/fe2b01873d92453599763e5523a862f8/users/97f1b1e52a984639a504441e3701b981/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.783911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b473dd-e2c7-478d-99f4-6fd93213101d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.789258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.789907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.790194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.790345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2b01873d92453599763e5523a862f8, user_id=97f1b1e52a984639a504441e3701b981, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.790480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.791214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.792793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.792997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.796074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:13.805059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 785/1583] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:13.805859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.815207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f15d94-1d15-433b-9fcd-bc91b1acda63 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.820205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.821209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.821548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.821827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.822497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.824366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.871641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 799/1584] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:13 2022] PUT /identity/v3/projects/fe2b01873d92453599763e5523a862f8/users/97f1b1e52a984639a504441e3701b981/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:13.872485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.898246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 786/1585] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.898246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:13.906904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca07d1c-5fb9-4ce3-a70d-8a912b3b1da7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.936184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a52753-a21e-48e7-991a-31661906ba75 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:13.941001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.941698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.941977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.942311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.942993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.944956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3907a3-33f0-42c2-a37d-eda176d8c58a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.964049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:13.964920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.965043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.965353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:13.966034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:13.967988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:13.997429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 800/1586] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:13.998544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:13.998835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:13.999115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.025836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] MFA Rules not processed for user `b4b57128eb75430b950c095737ac1822`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:14.027028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:14.054576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 787/1587] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:14.055425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.065413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c783fa-21bb-431a-bc0f-ca10ec6c7a81 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.070076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.070730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.071009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.071308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.072027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.073720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.084313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 788/1588] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.085182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.095322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eae82d-71c6-42fb-bd29-b0f14c7ae655 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.095591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 801/1589] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:14.096404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.103724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.104554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.104898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.105275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.106063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.107144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b8dc-bedb-4409-a6db-d471fe227ea3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.107839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01cd0b-2060-471c-8b8e-a715ad829a32 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.111913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.112779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.113062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.113368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] PATH_INFO: `/v3/projects/5bc2c939bdd545e2b241429dcd5026ea/users/6ec5573cc00f48a4805ce384eecf1e60/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.118987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 789/1590] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.120325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.120667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.120998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.130525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc2c939bdd545e2b241429dcd5026ea, user_id=6ec5573cc00f48a4805ce384eecf1e60, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.133104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.139271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:14.141048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:14.148849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 790/1591] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:14.149655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.158859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d091d16-3d7e-40d4-9b1c-b4a3c958e312 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.163469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.164108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.164393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.164672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.165426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.167005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.176843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 791/1592] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.177641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.186806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abb81-0a0d-4ba0-9768-b85a29de743e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.192005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 802/1593] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:14 2022] PUT /identity/v3/projects/5bc2c939bdd545e2b241429dcd5026ea/users/6ec5573cc00f48a4805ce384eecf1e60/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:14.198066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.244892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:14.246467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.247137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.247416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.247689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.248460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.250905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.256795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c02d-dff9-4c4d-a1c2-e1590f92a05e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.260765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 792/1594] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.261622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.264318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.265726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.266135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.266418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.267295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.268926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.275860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd10b2-0456-4fe4-adf2-31c7d2db007c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.280516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 803/1595] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.280719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.281250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.281497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.281806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.282178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.283624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.286341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.290907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116fc28-c3fb-4152-b8d9-b06a695730ab admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.296192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.296848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.297136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.297449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.298133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.300054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.305423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 793/1596] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.305423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.313273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:14.318325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ba2c8-6a06-4bb9-a958-32db2dac6265 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.323232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.324038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.324188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.324465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.325215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.326805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.342340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 794/1597] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.343133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.358327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 804/1598] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:14 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:14.361954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.400179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800c1eb-444e-4088-b239-2057b1dfed95 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.423401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8092ba-fd5a-4e68-b493-975f2fd2b1c0 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.428129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.428803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.429174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.429402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.430259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.431857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1e7f5-480c-42b3-a5b9-d3a1e90a916a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.442173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 805/1599] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.443315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.443611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.443887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.462974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.463705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.464075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.464358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.470511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] MFA Rules not processed for user `6ec5573cc00f48a4805ce384eecf1e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:14.471717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:14.474296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.475757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.486055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 795/1600] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:14.486922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.496099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8ffaa-b568-4153-92f8-de9abe5ad963 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.506132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db06531-baa8-4612-b651-75346065881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.506667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db06531-baa8-4612-b651-75346065881f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.506954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db06531-baa8-4612-b651-75346065881f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.507229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db06531-baa8-4612-b651-75346065881f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.508023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db06531-baa8-4612-b651-75346065881f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.510119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db06531-baa8-4612-b651-75346065881f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.521205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 796/1601] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.522114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db06531-baa8-4612-b651-75346065881f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.532344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db06531-baa8-4612-b651-75346065881f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.537083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.537753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.538038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.538316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.539027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.540607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.544534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 806/1602] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:14.545467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.550996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 797/1603] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:14.551836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.556694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f95910-2eb8-4cfb-8ba0-778aa8efeac2 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.561264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.562169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.562458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.562736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] PATH_INFO: `/v3/projects/165d7892df2f450d927c92d71cd96f29/users/83e06ab7cd924e07a79efaafa078a174/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.563055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e6bc7-2d19-436e-8040-d41f6b019db5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.567541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.568402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.568687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.568966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] PATH_INFO: `/v3/projects/3f550f8d11a74b8da5ad84607b79ac37/users/a760d437b597441fa9a4595995c5bc6f/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.578921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] RBAC: Authorizing `identity:create_grant(project_id=165d7892df2f450d927c92d71cd96f29, user_id=83e06ab7cd924e07a79efaafa078a174, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.581604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.584080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f550f8d11a74b8da5ad84607b79ac37, user_id=a760d437b597441fa9a4595995c5bc6f, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.585105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:14.585105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:39:14.585105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:39:14.587129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.649568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 807/1604] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:14 2022] PUT /identity/v3/projects/165d7892df2f450d927c92d71cd96f29/users/83e06ab7cd924e07a79efaafa078a174/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:14.650466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.679801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 798/1605] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:14 2022] PUT /identity/v3/projects/3f550f8d11a74b8da5ad84607b79ac37/users/a760d437b597441fa9a4595995c5bc6f/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:14.681823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.689645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:14.726957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a956-9ba1-4178-90f5-d662a0c2775a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.756183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa170-bc2e-4f21-9d8b-3cd2fce7b389 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.760870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.761803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.762085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.762356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] PATH_INFO: `/v3/projects/91e67f47a1ee45689cd5a315ce580ecb/users/326ecee0e4a445feb9f39335afc4c21c/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.774611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:14.778635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e67f47a1ee45689cd5a315ce580ecb, user_id=326ecee0e4a445feb9f39335afc4c21c, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.781997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.784191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.784861 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.785142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.785455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.786136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.788231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.879553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 799/1606] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:14 2022] PUT /identity/v3/projects/91e67f47a1ee45689cd5a315ce580ecb/users/326ecee0e4a445feb9f39335afc4c21c/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:14.880450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.937945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 808/1607] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:14 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:14.937945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:14.945556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf38b3-e4b8-49b3-9a57-09a07c3d7872 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.947326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9676f8-cb8a-4edc-b2a6-bd281cf1b29b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:14.950342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.950976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.951260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.951536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.952242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.952355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:14.953827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:14.954111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:14.954388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] PATH_INFO: `/v3/projects/98f3b8e5987e4daebdcfdd6311032a67/users/d2bb01bd22b14c418d6465441b0646d8/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:14.954735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:14.969625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f3b8e5987e4daebdcfdd6311032a67, user_id=d2bb01bd22b14c418d6465441b0646d8, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:14.972471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536df1-374a-4893-a38a-8596ca16e9be admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.044744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 809/1608] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:14 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:15.045724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.084948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 800/1609] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:14 2022] PUT /identity/v3/projects/98f3b8e5987e4daebdcfdd6311032a67/users/d2bb01bd22b14c418d6465441b0646d8/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:15.084948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174376-a619-4e30-acf9-83fbe144d929 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.084948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174376-a619-4e30-acf9-83fbe144d929 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.084948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174376-a619-4e30-acf9-83fbe144d929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.102100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-88174376-a619-4e30-acf9-83fbe144d929 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.103386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-88174376-a619-4e30-acf9-83fbe144d929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.111998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef39c72-6ac9-4fef-89c6-482ada0c853e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.165941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:15.173613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 801/1610] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.177734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28bc52-b244-42fa-b333-511f323ba14d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.177734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28bc52-b244-42fa-b333-511f323ba14d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.177734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28bc52-b244-42fa-b333-511f323ba14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.181379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.182538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.182958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.184226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] PATH_INFO: `/v3/projects/52569bf4eba24c2a9fa0ef8c26b8b22d/users/8f118c0ba8494e6e9f5a4488ebf8d220/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.195134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ba28bc52-b244-42fa-b333-511f323ba14d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.196360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ba28bc52-b244-42fa-b333-511f323ba14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.199715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52569bf4eba24c2a9fa0ef8c26b8b22d, user_id=8f118c0ba8494e6e9f5a4488ebf8d220, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.202449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.208308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 802/1611] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.208308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.208308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.208308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.231923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] MFA Rules not processed for user `326ecee0e4a445feb9f39335afc4c21c`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.233098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.288734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:15.295494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 810/1612] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:15 2022] PUT /identity/v3/projects/52569bf4eba24c2a9fa0ef8c26b8b22d/users/8f118c0ba8494e6e9f5a4488ebf8d220/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:15.296110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.303436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 803/1613] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.303436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.314719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016be24-df29-4dd6-9ffc-8b8312fbcfaf None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.378568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.379246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.379567 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.379838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.380526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.382517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.397336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d662e9-7bfa-4623-82bc-1b62cde15067 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.402078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.402769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.403055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.403334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.404059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.405725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d20223-57b5-4da2-b44c-74151187874e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.415881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 811/1614] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:15.418796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.423820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.423820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.452339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] MFA Rules not processed for user `d2bb01bd22b14c418d6465441b0646d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.453624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.477148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 804/1615] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:15.480436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.522193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 812/1616] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.523075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.524750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.535357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421b4e-cbb3-4d8c-90cb-84cb2a54e6b5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.544442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d72d0-0d41-4254-940b-8a9abbbd3be6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.544729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.544729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.544729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.544729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.545107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.545107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.545107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.545107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.552068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.553740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad7d7-a610-434c-8ca7-2b281f96eb6a service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.555806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.557489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4c5e8-1194-428e-8b30-2141d82ae253 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.565355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 805/1617] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:15.566664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.573101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.573101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.580958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 813/1618] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:15.582715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a46d-ffe9-400b-817f-b170cb884403 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.584242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a46d-ffe9-400b-817f-b170cb884403 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.586199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.586651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815a46d-ffe9-400b-817f-b170cb884403 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.587167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.588475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 814/1619] 10.222.0.30 () {54 vars in 961 bytes} [Tue Jul 12 14:39:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:15.590228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815a46d-ffe9-400b-817f-b170cb884403 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.602013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 806/1620] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.603055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.614856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf24b6-79e1-4e95-b840-344a535e518a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.621122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.623453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815a46d-ffe9-400b-817f-b170cb884403 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.623578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.623578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.623578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.623578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.625458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42414cf-fa72-4c5d-b6d2-9ed9387a98d5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.628640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.629781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.630051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.631046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] PATH_INFO: `/v3/projects/21e91c44cec34af6883bd4f99ff11aad/users/eaf6da2e26de403788e4821bf051df8a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.640753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 807/1621] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:15.640753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a13-d570-4381-9a77-844d48108914 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.640753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a13-d570-4381-9a77-844d48108914 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.640753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9045a13-d570-4381-9a77-844d48108914 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.642431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 808/1622] 10.222.0.30 () {54 vars in 961 bytes} [Tue Jul 12 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:15.643446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.644097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.644251 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.656111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e91c44cec34af6883bd4f99ff11aad, user_id=eaf6da2e26de403788e4821bf051df8a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.659048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.683519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.684654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.734426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:15.752108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 809/1623] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.754065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.766098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 815/1624] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:15 2022] PUT /identity/v3/projects/21e91c44cec34af6883bd4f99ff11aad/users/eaf6da2e26de403788e4821bf051df8a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:15.768631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.778228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb393d-13e7-4d94-a2a7-35765fc9d0ab None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.796837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:15.832971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.833690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.833956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.834236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.841719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef15d7a-9230-4d62-bc55-87e53aa13fbc admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:15.844216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.845723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2ceb3-92ae-4104-b006-594c2bfa1734 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.847449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:15.848876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.849402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.854093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] PATH_INFO: `/v3/projects/6f702d425dad4d648c13bca3da0b05d8/users/3444303e294742798c3570f0f6fc0e39/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.856281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 810/1625] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:15.857724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:15.857860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:15.858100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:15.877084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:15.878340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:15.879349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f702d425dad4d648c13bca3da0b05d8, user_id=3444303e294742798c3570f0f6fc0e39, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:15.895553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:15.971942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 811/1626] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Jul 12 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:15.972685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:15.989596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f79e-2d22-4189-ade2-89c4d957e020 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.005773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.006608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.006903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.007025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.033091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 816/1627] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:15 2022] PUT /identity/v3/projects/6f702d425dad4d648c13bca3da0b05d8/users/3444303e294742798c3570f0f6fc0e39/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:16.037969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.113941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.116392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.131885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c463a-350e-4b2e-9266-3d8cea627b36 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.137642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.138319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.138657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.138937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.139075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 812/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:16.140447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.142076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.143599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.157122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 817/1629] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:16.159011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.177199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8709-6a83-402d-98a9-22bd3f319280 service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.180957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41cc5-7702-4cf5-af6c-ef3145ee2a97 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.194454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.195255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.195514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.195844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.196653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.200486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.238256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 818/1630] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:16.239072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.264392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd0795-cf30-4022-b90d-3836fe70f97e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.278793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.279887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.280144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.280552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] PATH_INFO: `/v3/projects/21e91c44cec34af6883bd4f99ff11aad/users/eaf6da2e26de403788e4821bf051df8a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.284024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e91c44cec34af6883bd4f99ff11aad, user_id=eaf6da2e26de403788e4821bf051df8a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.287878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.299020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.302746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.302746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.302746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.303054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 819/1631] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:16 2022] PUT /identity/v3/projects/21e91c44cec34af6883bd4f99ff11aad/users/eaf6da2e26de403788e4821bf051df8a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:16.303054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.432987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.443168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e13cd-0b7b-4097-b817-558d87c3f79c service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.450329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b02020-e187-464c-9747-d1607c68456a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.472123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.472123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.472123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.472123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] PATH_INFO: `/v3/projects/6f702d425dad4d648c13bca3da0b05d8/users/3444303e294742798c3570f0f6fc0e39/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.472432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 813/1632] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:16.472432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.472432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.472432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.476730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f702d425dad4d648c13bca3da0b05d8, user_id=3444303e294742798c3570f0f6fc0e39, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.480654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d44a5-7934-4aaf-81c2-76d3cdbf0da4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.513100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] MFA Rules not processed for user `eaf6da2e26de403788e4821bf051df8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:16.514749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:16.533105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 820/1633] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:16 2022] PUT /identity/v3/projects/6f702d425dad4d648c13bca3da0b05d8/users/3444303e294742798c3570f0f6fc0e39/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:16.543268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.543567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.546178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.549508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:16.595368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] MFA Rules not processed for user `3444303e294742798c3570f0f6fc0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:16.598473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:16.656938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 814/1634] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:16.674669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.690829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:16.707954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 821/1635] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:16.726446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:16.810200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7033b63-29a8-4ede-a2ad-9076e4beec11 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.810447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1595c2d-7f6e-449f-96c0-4b9433350e53 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:16.814910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.815778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.815969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.816229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.821299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:16.821979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.822319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.822597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.835763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:16.837138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.840168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4efa0-dbfe-4ff5-84c2-6066ddce64ae service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.851280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:16.852851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b126e47-92a4-42fc-a5c6-c65e6d638bec service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:16.853728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 822/1636] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:16.855731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3701-d34b-4632-8323-89d622c0a9a6 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.856034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3701-d34b-4632-8323-89d622c0a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.856316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3701-d34b-4632-8323-89d622c0a9a6 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.857907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 823/1637] 10.222.0.30 () {54 vars in 912 bytes} [Tue Jul 12 14:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:16.866362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.866826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.867127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.888647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 815/1638] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:16.914955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:16.916246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:16.961822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133028ae-7a95-45a5-908e-0a21e49b64e5 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.963112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133028ae-7a95-45a5-908e-0a21e49b64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.967048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133028ae-7a95-45a5-908e-0a21e49b64e5 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:16.971385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 816/1639] 10.222.0.30 () {54 vars in 912 bytes} [Tue Jul 12 14:39:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:39:16.977732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:16.978349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:16.978978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:17.022529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:17.024502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:17.094564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 824/1640] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:17.112875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:17.127427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee716f6-4081-4c4a-89b9-c678fc662a8b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:17.140263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:17.142493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:17.143080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:17.143552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:17.167107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 817/1641] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:17.191988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:17.193483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:17.206845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 825/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:17.322335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:17.329292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:17.337420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4786b9-ee73-449e-b5b4-cd374f8d6b75 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:17.341410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1ab9-dbd5-48ba-b965-da5cb6f27c63 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:17.348395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:17.349063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:17.349381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:17.349659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:17.354165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:17.354843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:17.357257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:17.357573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:17.377093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:17.394991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:17.418262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 826/1643] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:17.447644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:17.449737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432eaca-5842-4412-8d0c-efc7f729a29b service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:17.467215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 818/1644] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:18.363832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:18.369168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:18.510274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ec381-6227-4545-9751-e7e932c9ede7 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:18.522262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:18.523230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:18.523572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:18.523780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] PATH_INFO: `/v3/users/326ecee0e4a445feb9f39335afc4c21c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:18.525048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326ecee0e4a445feb9f39335afc4c21c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:18.527040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:18.823153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326ecee0e4a445feb9f39335afc4c21c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:18.823153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326ecee0e4a445feb9f39335afc4c21c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:18.823153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326ecee0e4a445feb9f39335afc4c21c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:18.836995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326ecee0e4a445feb9f39335afc4c21c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:18.857639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326ecee0e4a445feb9f39335afc4c21c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:18.947512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 827/1645] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:18 2022] DELETE /identity/v3/users/326ecee0e4a445feb9f39335afc4c21c => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:18.951957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:18.953074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:18.953712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.003904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:19.009569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.162473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 819/1646] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:19.178847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:19.180138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.312856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:19.314395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c749-9e3c-4037-a230-ed89463ee726 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:19.315849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.329046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:19.329046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.329405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.329639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.341905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cb6a5-46d3-4f35-ac6f-fd15bb549dbe None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:19.356867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:19.359186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:19.359696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.360117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.360517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] PATH_INFO: `/v3/users/6ec5573cc00f48a4805ce384eecf1e60` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.360634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6b562-58df-4fab-95b3-730033f935c6 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:19.363008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec5573cc00f48a4805ce384eecf1e60)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:19.366186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: WARNING py.warnings [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:19.366186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: warnings.warn(msg) Jul 12 14:39:19.366186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: Jul 12 14:39:19.368836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:19.385287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 828/1647] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:19.394435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:19.601840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec5573cc00f48a4805ce384eecf1e60'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.603990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec5573cc00f48a4805ce384eecf1e60'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.604674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec5573cc00f48a4805ce384eecf1e60'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.623182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec5573cc00f48a4805ce384eecf1e60'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.657826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec5573cc00f48a4805ce384eecf1e60'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.805777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 820/1648] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:19 2022] DELETE /identity/v3/users/6ec5573cc00f48a4805ce384eecf1e60 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:19.810417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.810417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.810657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.846383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:19.847631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.883364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:19.966877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 829/1649] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:20.011438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.017009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:20.228934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001176d-199c-4a6a-94f8-11892814b027 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.251768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.251954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.252253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.252610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.282039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.285990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e3e99-2ea1-429a-8a2b-5f2e32104d3e service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.323644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 821/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:20.482582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.485503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:20.513705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fe2c9-ef1a-4476-927c-8d8e0b238f80 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.524863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.525516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.525798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.526080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] PATH_INFO: `/v3/projects/91e67f47a1ee45689cd5a315ce580ecb` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.527512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e67f47a1ee45689cd5a315ce580ecb)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.528546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: WARNING py.warnings [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:39:20.528546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: warnings.warn(msg) Jul 12 14:39:20.528546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: Jul 12 14:39:20.536071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.672192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e67f47a1ee45689cd5a315ce580ecb', 'request_id': 'req-850e6305-88ff-4ff6-835e-852990f07ed8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:20.680319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e67f47a1ee45689cd5a315ce580ecb', 'request_id': 'req-850e6305-88ff-4ff6-835e-852990f07ed8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:20.804789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e67f47a1ee45689cd5a315ce580ecb', 'request_id': 'req-850e6305-88ff-4ff6-835e-852990f07ed8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:20.845905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] The token cache is being invalidate because project 91e67f47a1ee45689cd5a315ce580ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:20.846722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:20.848151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-850e6305-88ff-4ff6-835e-852990f07ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:20.851064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 830/1651] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:20 2022] DELETE /identity/v3/projects/91e67f47a1ee45689cd5a315ce580ecb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:20.856709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.856930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.857190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.880945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:20.883131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:20.988399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.988954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.989534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.008998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 822/1652] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:21.035962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:21.041589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:21.079832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.113537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 831/1653] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:21.114501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c53e9-b927-4c77-be18-46e20488dfb2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.123255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.124999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.129785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.130204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.131336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:21.132280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.134247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.162965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 823/1654] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:21.164244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.179389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23223-5a63-44e9-871e-fdbcf3666495 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.194212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.195115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.195536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.196066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.196872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.199235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.356924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 832/1655] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:21 2022] POST /identity/v3/projects => generated 405 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:21.361543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.384307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa06c2-2046-4fdf-bba9-878afd422efc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.506488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:21.710974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.711919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.712376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.712793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.728104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.730462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.849954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 824/1656] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:21 2022] POST /identity/v3/users => generated 303 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:21.853736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.882815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce066472-b196-456f-9d7f-c4156df32af1 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.912807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.913117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.913427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.913707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.914455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.916108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.960930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 833/1657] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:21.962993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.977905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc689a-017d-4ebc-931f-355f00db94da admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.022362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.023545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.023989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.024281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.025044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.029824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ac037-1ba2-4ec0-a8de-bb340ff11149 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.057555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 825/1658] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:22.061643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.076538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e011e4-87e0-4117-8cd4-faf2f698a339 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.084626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.086121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.086447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.086751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] PATH_INFO: `/v3/projects/a2a5a85756714784bef24ea23bcf7dac/users/490fcc616c914fd0a12ba25bfda0bc78/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.119390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a5a85756714784bef24ea23bcf7dac, user_id=490fcc616c914fd0a12ba25bfda0bc78, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.139358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.247112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 834/1659] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:22 2022] PUT /identity/v3/projects/a2a5a85756714784bef24ea23bcf7dac/users/490fcc616c914fd0a12ba25bfda0bc78/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:22.257473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.266410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.267969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.339724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] MFA Rules not processed for user `490fcc616c914fd0a12ba25bfda0bc78`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:22.341019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.385944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.388676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.479042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 826/1660] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:22.490181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.625308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:22.634393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c084ec-9435-4645-b0a1-b5b1d9d37ec9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.651994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.653018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.653527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.653844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.654667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.656930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.680619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a0eac-ab00-4ddf-8957-9b8f941d716a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.693692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.700415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.700590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.700817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] PATH_INFO: `/v3/projects/5bc2c939bdd545e2b241429dcd5026ea` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.702804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc2c939bdd545e2b241429dcd5026ea)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.704811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.814043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 827/1661] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:22 2022] POST /identity/v3/projects => generated 405 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:22.820508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.881759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc2c939bdd545e2b241429dcd5026ea', 'request_id': 'req-874dffae-0b32-4a6d-a29f-095b99c32e6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.907420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc2c939bdd545e2b241429dcd5026ea', 'request_id': 'req-874dffae-0b32-4a6d-a29f-095b99c32e6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.018358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc2c939bdd545e2b241429dcd5026ea', 'request_id': 'req-874dffae-0b32-4a6d-a29f-095b99c32e6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.020158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed633-92b3-4fd2-9782-9f52b9727de7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.042610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] The token cache is being invalidate because project 5bc2c939bdd545e2b241429dcd5026ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:23.042916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.048624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.059761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 835/1662] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:22 2022] DELETE /identity/v3/projects/5bc2c939bdd545e2b241429dcd5026ea => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:23.060771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.065493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:23.137487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.138525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.140936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.140936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.140936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.143276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.267810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dffae-0b32-4a6d-a29f-095b99c32e6c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.268574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:23.270733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 828/1663] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:22 2022] POST /identity/v3/users => generated 309 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:23.274423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.285461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:23.286528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.287626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.287738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.287985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] PATH_INFO: `/v3/users/3444303e294742798c3570f0f6fc0e39` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.300496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3444303e294742798c3570f0f6fc0e39)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.305003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.330118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc458-9734-490b-a326-4db357336093 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.354573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.355397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.355785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.356166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.357083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.363992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.389312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 829/1664] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.393263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.419252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4e424-038b-4d33-91dd-d0011ceeef83 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.437073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.442519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.443041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.443515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] PATH_INFO: `/v3/projects/481aec0c927e4e059113f24ad139ce43/users/b8bc95936bd64fe9b559c0921fb1c55e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.477252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] RBAC: Authorizing `identity:create_grant(project_id=481aec0c927e4e059113f24ad139ce43, user_id=b8bc95936bd64fe9b559c0921fb1c55e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.488146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.576442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 830/1665] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:23 2022] PUT /identity/v3/projects/481aec0c927e4e059113f24ad139ce43/users/b8bc95936bd64fe9b559c0921fb1c55e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:23.580851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.586008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3444303e294742798c3570f0f6fc0e39'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.586931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3444303e294742798c3570f0f6fc0e39'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.587741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3444303e294742798c3570f0f6fc0e39'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.604890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3444303e294742798c3570f0f6fc0e39'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.632257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3444303e294742798c3570f0f6fc0e39'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:23.715841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e89a84-0ae7-4bc6-a201-5e968ff9143e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.724127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.724815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.725099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.725401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.726688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 836/1666] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:23 2022] DELETE /identity/v3/users/3444303e294742798c3570f0f6fc0e39 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:23.728084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.730434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98827f-c045-407d-8353-25fc124bf513 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.744410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 831/1667] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.748407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.752371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:23.910089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d77a98-e913-4a6c-b6ef-f88e03b57eb2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.934871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.935268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.935550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.935901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] PATH_INFO: `/v3/projects/481aec0c927e4e059113f24ad139ce43/users/b8bc95936bd64fe9b559c0921fb1c55e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.942848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=481aec0c927e4e059113f24ad139ce43, user_id=b8bc95936bd64fe9b559c0921fb1c55e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.949007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:24.003674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 837/1668] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:23 2022] PUT /identity/v3/projects/481aec0c927e4e059113f24ad139ce43/users/b8bc95936bd64fe9b559c0921fb1c55e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:24.015413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:24.015703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:24.015983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:24.060425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] MFA Rules not processed for user `b8bc95936bd64fe9b559c0921fb1c55e`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:24.064308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:24.229330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 832/1669] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:24.241341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:24.243380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:24.433236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775b6c2-6ea3-4e17-8370-9d92ad20cc92 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:24.444644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:24.447437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:24.447731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:24.448012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:24.471801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:24.473578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4004823-94de-4779-9ec2-9c29715e28bc service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:24.485756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 838/1670] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:24.626879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:24.654347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:25.031585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:25.033881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:25.189752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d39243-5ef2-4876-9d6a-4555b2e0daea None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:25.202431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:25.203553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.203939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.204320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] PATH_INFO: `/v3/projects/6f702d425dad4d648c13bca3da0b05d8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.206118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f702d425dad4d648c13bca3da0b05d8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:25.210715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:25.376529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f702d425dad4d648c13bca3da0b05d8', 'request_id': 'req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:25.392306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f702d425dad4d648c13bca3da0b05d8', 'request_id': 'req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:25.466743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f702d425dad4d648c13bca3da0b05d8', 'request_id': 'req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:25.504554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] The token cache is being invalidate because project 6f702d425dad4d648c13bca3da0b05d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:25.504920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:25.507757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d3ed7afd-e2a4-4669-8650-e1c06487c6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:25.514361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 833/1671] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:25 2022] DELETE /identity/v3/projects/6f702d425dad4d648c13bca3da0b05d8 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:25.516055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.516525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.516631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.558448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:25.559713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:25.706573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 839/1672] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:25.714328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.714825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.714943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.757973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:25.759319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:25.776093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 834/1673] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:25.776902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:25.790460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060140c-3272-4732-9ecd-a23b30e6f7b9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:25.797869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:25.798573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.799005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.799140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.800080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:25.800832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:25.820188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:25.845839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 840/1674] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:25.849879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:25.872728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ab13-6202-47f4-baf7-a5f52702a1ec None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:25.879686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:25.880379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.880666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.880959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.882530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:25.884804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:25.941683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:25.943729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:25.971377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1358f-bea6-4202-a91e-5e63439c8f41 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:25.984957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:25.985693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:25.985946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:25.986291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] PATH_INFO: `/v3/users/490fcc616c914fd0a12ba25bfda0bc78` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:25.987804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490fcc616c914fd0a12ba25bfda0bc78)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:25.994728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.034228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:26.053494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 835/1675] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:25 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:26.057788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.174840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:26.222628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad367-b56c-4cf2-a0ad-49f1c33b5414 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.234173 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:26.234890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.236320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.236619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.237361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:26.240111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.330718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490fcc616c914fd0a12ba25bfda0bc78'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:26.334303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490fcc616c914fd0a12ba25bfda0bc78'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:26.337298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490fcc616c914fd0a12ba25bfda0bc78'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:26.356813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490fcc616c914fd0a12ba25bfda0bc78'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:26.378243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 836/1676] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:26 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:26.378625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.380953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:26.391330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4888262b-774c-44eb-9839-e1d786733fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490fcc616c914fd0a12ba25bfda0bc78'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:26.417068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac335c-0113-401d-b89a-eeacfe1850b4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.428801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:26.429530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.429814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.430093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.437916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:26.444554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.473805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 837/1677] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:26.481909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.496895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295e1a-6732-4b4c-a692-eb0bdccba24b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.506049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:26.506895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.507837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.508450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.509235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:26.512221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.541996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 841/1678] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:25 2022] DELETE /identity/v3/users/490fcc616c914fd0a12ba25bfda0bc78 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:26.549276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.558351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.560776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.606213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 838/1679] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:26.607376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.630343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:26.632547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc44d69-61e2-4bb0-94fd-782ba467a7b4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.634183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:26.757195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:26.758127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.758405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.758688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] PATH_INFO: `/v3/projects/b8dac4a3c40747b1970dc9b73b237a56/users/55b13cfb5c3541fbbf631fd5621518c3/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.786681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 842/1680] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:26.790106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dac4a3c40747b1970dc9b73b237a56, user_id=55b13cfb5c3541fbbf631fd5621518c3, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:26.792837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2386f-322a-4aad-a9d9-001bc5bc34cc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.816801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.818455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:26.884003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 839/1681] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:26 2022] PUT /identity/v3/projects/b8dac4a3c40747b1970dc9b73b237a56/users/55b13cfb5c3541fbbf631fd5621518c3/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:26.896021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.896316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.896600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.984923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfdb33-83b5-4746-bb9c-b0fdc54f518d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.986164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] MFA Rules not processed for user `55b13cfb5c3541fbbf631fd5621518c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:26.987557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:27.136898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.139057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.139057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.139932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.157441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 840/1682] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:27.161797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:27.303886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f6910-100c-45db-83c2-d9874b6e806b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:27.312175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.312718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.313954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.314172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.314882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:27.316851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:27.328974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:27.330476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0abd5-8ee8-4827-97cc-ec3f771b8807 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:27.345995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 843/1683] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:27.382305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.382659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.382989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.383821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:27.416406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] MFA Rules not processed for user `8f118c0ba8494e6e9f5a4488ebf8d220`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:27.418262 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:27.444413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:27.480085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 841/1684] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:27 2022] POST /identity/v3/projects => generated 363 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:27.485347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:27.543086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 844/1685] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:27.552345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:27.558786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:27.661014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2c2c6-b726-4c19-b7b8-48e1dbf4016b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:27.666753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.667336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.667458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.668186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.669358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:27.672210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:27.742013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a1d5-8dd9-485f-bbfb-34c1f6a076d6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:27.821101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 842/1686] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:27 2022] POST /identity/v3/users => generated 288 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:27.829526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:27.864648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05235a-76be-4bcc-9f96-ca1d0ec13aeb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:27.880067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.881132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.882926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.883304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.886171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:27.887873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:27.936223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.939749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.942496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.942612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 843/1687] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:27.942709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:27.950096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:27.964892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ba1e8-4218-431b-8bd4-f29de77b35de admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:27.977072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:27.979758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:27.980059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:27.980344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] PATH_INFO: `/v3/projects/bbd513ca99a9472f8309b7c1310f7f87/users/a660528a466343aeb54a551bc755ce8d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.007926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd513ca99a9472f8309b7c1310f7f87, user_id=a660528a466343aeb54a551bc755ce8d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.012025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.084755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.087123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.118132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 845/1688] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:28.130235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 844/1689] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:27 2022] PUT /identity/v3/projects/bbd513ca99a9472f8309b7c1310f7f87/users/a660528a466343aeb54a551bc755ce8d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:28.134328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.159433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b1bfa-bdca-4c53-a902-e63705268e3a service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.278223 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.281409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.296394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.297278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.297666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.298872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.306839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.314564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.315200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251c8ee-44df-4ae3-aab6-bf57c49bedd6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.334977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.336230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.336689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.337194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] PATH_INFO: `/v3/projects/a2a5a85756714784bef24ea23bcf7dac` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.338721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a5a85756714784bef24ea23bcf7dac)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.339464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 846/1690] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:28.341390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.347548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.369118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:28.374996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971768a8-0b9f-48b0-9ae8-2058d6f83873 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.389468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.389709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.389992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.390265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] PATH_INFO: `/v3/projects/bbd513ca99a9472f8309b7c1310f7f87/users/a660528a466343aeb54a551bc755ce8d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.397691 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd513ca99a9472f8309b7c1310f7f87, user_id=a660528a466343aeb54a551bc755ce8d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.407926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f7ba7-d61d-41a0-a9be-5f552bcd909d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.447760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 847/1691] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:28 2022] PUT /identity/v3/projects/bbd513ca99a9472f8309b7c1310f7f87/users/a660528a466343aeb54a551bc755ce8d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:28.456439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.456990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.458748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.502605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] MFA Rules not processed for user `a660528a466343aeb54a551bc755ce8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:28.503974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.539237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a5a85756714784bef24ea23bcf7dac', 'request_id': 'req-bed3522c-19a0-44e6-92d9-7617b92036f7'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.563552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a5a85756714784bef24ea23bcf7dac', 'request_id': 'req-bed3522c-19a0-44e6-92d9-7617b92036f7'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.660495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a5a85756714784bef24ea23bcf7dac', 'request_id': 'req-bed3522c-19a0-44e6-92d9-7617b92036f7'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.672107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 848/1692] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:28.679902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.682458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:28.684419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] The token cache is being invalidate because project a2a5a85756714784bef24ea23bcf7dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:28.684419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.684748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.685074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.690109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 845/1693] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:28 2022] DELETE /identity/v3/projects/a2a5a85756714784bef24ea23bcf7dac => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:28.700640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.700640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.873787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da713e-384e-426a-a608-80dcbb18c07f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.910386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3522c-19a0-44e6-92d9-7617b92036f7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.930066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.931953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.933261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.933603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] PATH_INFO: `/v3/users/b8bc95936bd64fe9b559c0921fb1c55e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.940649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bc95936bd64fe9b559c0921fb1c55e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.950105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.015460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:29.027374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.027374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.027374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.027374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.030553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.217473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.218959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.229584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bc95936bd64fe9b559c0921fb1c55e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.230338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bc95936bd64fe9b559c0921fb1c55e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.231013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bc95936bd64fe9b559c0921fb1c55e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.235764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 849/1694] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:29.239935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bc95936bd64fe9b559c0921fb1c55e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.271963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c117ec28-91cb-46a4-acd4-cb48ee76eee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bc95936bd64fe9b559c0921fb1c55e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.400341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 846/1695] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:28 2022] DELETE /identity/v3/users/b8bc95936bd64fe9b559c0921fb1c55e => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:29.499540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.502087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.516710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a3579-f76e-49e3-9ce2-3b8e33c5d359 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:29.627237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.628138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.628566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.629115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.632563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:29.637512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.711127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.711466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.711710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.771749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.773962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c508-9916-4f32-9098-710f836e1de2 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.785029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] MFA Rules not processed for user `97f1b1e52a984639a504441e3701b981`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:29.786377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.790877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 850/1696] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:29.810273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:29.972561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 847/1697] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:29.982456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.983311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.983589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.026273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] MFA Rules not processed for user `97f1b1e52a984639a504441e3701b981`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:30.028007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.037908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 851/1698] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:30.048891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.050677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.202156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af92ef7-9c7a-4498-982e-4c45af06b737 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.209885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.211107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.211107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.211284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.231707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.241076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f991b-59d5-4b88-803c-a826cc8b6a32 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.272721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 848/1699] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:30.678085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.679552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.830195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be2f1-bd76-4a85-85b4-252a23976e1b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.842828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.843730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.844011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.844286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] PATH_INFO: `/v3/projects/481aec0c927e4e059113f24ad139ce43` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.847195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=481aec0c927e4e059113f24ad139ce43)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.850446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.140856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481aec0c927e4e059113f24ad139ce43', 'request_id': 'req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:31.168426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481aec0c927e4e059113f24ad139ce43', 'request_id': 'req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:31.282733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481aec0c927e4e059113f24ad139ce43', 'request_id': 'req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:31.309305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] The token cache is being invalidate because project 481aec0c927e4e059113f24ad139ce43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:31.311374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:31.314565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6290197d-b3c3-4d4d-8ec5-a8f623140f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:31.320001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 852/1700] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:30 2022] DELETE /identity/v3/projects/481aec0c927e4e059113f24ad139ce43 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:31.324393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716e90-2970-4a57-af3d-a99b82baae27 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.325375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716e90-2970-4a57-af3d-a99b82baae27 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.325852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716e90-2970-4a57-af3d-a99b82baae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.374687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2a716e90-2970-4a57-af3d-a99b82baae27 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:31.376296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2a716e90-2970-4a57-af3d-a99b82baae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.532813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 849/1701] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:31.541823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.542258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.542611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.609993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:31.622392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.625060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2136c-37a0-4735-a85f-14c953c32cb9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.625550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2136c-37a0-4735-a85f-14c953c32cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.626025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2136c-37a0-4735-a85f-14c953c32cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.648583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 853/1702] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:31.654136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.685540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f68ad1-a336-4d31-a026-23f58d886ebd None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.713520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.714210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.714493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.714769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.717581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b8d2136c-37a0-4735-a85f-14c953c32cb9 None None] MFA Rules not processed for user `a760d437b597441fa9a4595995c5bc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:31.718829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b8d2136c-37a0-4735-a85f-14c953c32cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.727487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:31.727821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.729547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.744455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:31.779911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 854/1703] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:31.784309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.829227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6c13-b4a5-4b9f-bd85-17758725fae6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.830462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:31.848336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.848903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.849610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.850696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.851422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.853417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.956791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 850/1704] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:31.965529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963fd8-b2a2-460b-aab4-9ef04b0a06ef None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.965529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963fd8-b2a2-460b-aab4-9ef04b0a06ef None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.965529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963fd8-b2a2-460b-aab4-9ef04b0a06ef None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.968186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 851/1705] 10.222.0.30 () {54 vars in 912 bytes} [Tue Jul 12 14:39:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:31.969650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.973997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.974541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.998899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 855/1706] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:31.998899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.007432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.125817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:32.127612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.212181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34dc2-4d11-4fc6-8d3b-bb293adf3461 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.219522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.220278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.220566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.220981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.300038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 852/1707] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:32.301206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.331918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.333646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.360653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 856/1708] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:32.362940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.370003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.413935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5f7-216e-4ce5-b48b-5b68b8751227 service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.429349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.430874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.431169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.431491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.455774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.461204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.479342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 857/1709] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:32.498877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcc1cc-0918-4f66-b46b-36024e094695 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.519470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.520168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.520458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.520742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.521487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.526669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.640403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 853/1710] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:32 2022] POST /identity/v3/users => generated 291 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:32.645767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.667148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18afbd-6f25-4d4e-845e-4b0f0b1a6fb0 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.682429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.684189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.684372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.684655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.695900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.703123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.738327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 858/1711] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:32.742372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.764404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d186-5b72-4c5c-a306-6bf410de56a2 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.777819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.778785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.779233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.780368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.781361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.783402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.827019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 854/1712] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:32.830981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.859456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0fcd-b639-42a6-8bd4-6c27eda5f285 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.878145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.879079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.879361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.879639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] PATH_INFO: `/v3/projects/5cd73adf4ec049ceb284156dd16ccf04/users/2cd5465169634b85bbd56e49fd738f5e/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.947478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd73adf4ec049ceb284156dd16ccf04, user_id=2cd5465169634b85bbd56e49fd738f5e, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.973508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb7b85-083c-4e93-b6c6-c49223f869bd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.024913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.027127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.058630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474761f3-59a6-417f-9c5f-bb9cfb5721c6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.068089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.069432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.069832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.070441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] PATH_INFO: `/v3/users/d2bb01bd22b14c418d6465441b0646d8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.072042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2bb01bd22b14c418d6465441b0646d8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.073394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 859/1713] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:32 2022] PUT /identity/v3/projects/5cd73adf4ec049ceb284156dd16ccf04/users/2cd5465169634b85bbd56e49fd738f5e/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:33.074818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.085800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380952f8-f036-4c03-8d17-3459f246122e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.086411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380952f8-f036-4c03-8d17-3459f246122e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.086699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380952f8-f036-4c03-8d17-3459f246122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.206022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-380952f8-f036-4c03-8d17-3459f246122e None None] MFA Rules not processed for user `2cd5465169634b85bbd56e49fd738f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.207266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-380952f8-f036-4c03-8d17-3459f246122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.322967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:33.374445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2bb01bd22b14c418d6465441b0646d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.376192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2bb01bd22b14c418d6465441b0646d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.377728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2bb01bd22b14c418d6465441b0646d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.384853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 860/1714] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:33.389835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380952f8-f036-4c03-8d17-3459f246122e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.405497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2bb01bd22b14c418d6465441b0646d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.453472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2e5b010b-af4d-4b73-92dc-c30c083d2aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2bb01bd22b14c418d6465441b0646d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.493925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:33.536742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 855/1715] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:33 2022] DELETE /identity/v3/users/d2bb01bd22b14c418d6465441b0646d8 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:33.545738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19357-5091-443d-8b44-3ef8e3e344f5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.546265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19357-5091-443d-8b44-3ef8e3e344f5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.546744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19357-5091-443d-8b44-3ef8e3e344f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.588763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380952f8-f036-4c03-8d17-3459f246122e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.606578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-09c19357-5091-443d-8b44-3ef8e3e344f5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.610483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-09c19357-5091-443d-8b44-3ef8e3e344f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.778875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.782254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.790322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.790886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.791797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.796405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.851185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 856/1716] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:33.887680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5d23-7837-4570-aa42-5777e9708476 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.888057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5d23-7837-4570-aa42-5777e9708476 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.890017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5d23-7837-4570-aa42-5777e9708476 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.891527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 857/1717] 10.222.0.30 () {54 vars in 960 bytes} [Tue Jul 12 14:39:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:33.896984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.897384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.897861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.960927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 861/1718] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/projects => generated 381 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:33.963674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.965414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.979035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.981760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.201755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc55d29-9c4f-484e-bcc2-f0af4124b93c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.210680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.211432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.211647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.214758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.215522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.218206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.227659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 858/1719] 10.222.0.30 () {58 vars in 1057 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:34.228742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.231695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.339654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 862/1720] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/users => generated 297 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:34.340975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.382402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b436828-0828-499d-bd1e-0935119ca109 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.395812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.396493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.396951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.397263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.400851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.407440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbb3e5-20ca-4bf7-b7bc-f895d52121fc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.412913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.413865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.414299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.414944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.439160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.442138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b1bfc-39fd-4050-8626-1df735a6a7ba service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.452692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.454565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.463041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 859/1721] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:34.464990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb035-2dd7-4361-9610-804a91797dab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.465468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb035-2dd7-4361-9610-804a91797dab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.467200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb035-2dd7-4361-9610-804a91797dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.488191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 863/1722] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:34.489115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.496017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-966bb035-2dd7-4361-9610-804a91797dab None None] MFA Rules not processed for user `97f1b1e52a984639a504441e3701b981`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:34.497635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-966bb035-2dd7-4361-9610-804a91797dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.528005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb97f8-eb9f-493e-a635-5ae880f9bf6f service placement] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.538669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.539400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.539627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.539900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.540635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.542285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a25aa6-c2f1-461c-9c4b-c9dba85832eb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.586316 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 864/1723] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:34.587827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f088e7-0ae6-420b-ac49-03aa5f21de50 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.588336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f088e7-0ae6-420b-ac49-03aa5f21de50 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.588616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f088e7-0ae6-420b-ac49-03aa5f21de50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.628888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-49f088e7-0ae6-420b-ac49-03aa5f21de50 None None] MFA Rules not processed for user `8f118c0ba8494e6e9f5a4488ebf8d220`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:34.632471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-49f088e7-0ae6-420b-ac49-03aa5f21de50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.646919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 860/1724] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:34.649147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb035-2dd7-4361-9610-804a91797dab None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.692510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb035-2dd7-4361-9610-804a91797dab None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.716189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.717184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.717489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.717768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] PATH_INFO: `/v3/projects/ccc937f4c3b64b4494939198c3b14c47/users/33d51aaf5c4a4a9ab9412a11f103466a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.757945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc937f4c3b64b4494939198c3b14c47, user_id=33d51aaf5c4a4a9ab9412a11f103466a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.763116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.828623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 865/1725] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:34.893362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b52053-fb53-47a3-9c8b-9ddf722f6c76 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.893362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b52053-fb53-47a3-9c8b-9ddf722f6c76 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.893542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b52053-fb53-47a3-9c8b-9ddf722f6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.911744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 861/1726] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:34 2022] PUT /identity/v3/projects/ccc937f4c3b64b4494939198c3b14c47/users/33d51aaf5c4a4a9ab9412a11f103466a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:34.918320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.931796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b9b52053-fb53-47a3-9c8b-9ddf722f6c76 None None] MFA Rules not processed for user `83e06ab7cd924e07a79efaafa078a174`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:34.933060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b9b52053-fb53-47a3-9c8b-9ddf722f6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.065721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.118021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52659c-c93b-494e-9c91-2ccc358d91e5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.130532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.131284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.131575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.131857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.133519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.135650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.160108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 862/1727] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:35.164136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.232051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8e85-6ad9-4898-b4a4-8bd0add2d0ac admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.253204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 866/1728] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.261754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.262782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.266416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.266877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.266981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.267076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] PATH_INFO: `/v3/projects/ccc937f4c3b64b4494939198c3b14c47/users/33d51aaf5c4a4a9ab9412a11f103466a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.267159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.275354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc937f4c3b64b4494939198c3b14c47, user_id=33d51aaf5c4a4a9ab9412a11f103466a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.291781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d86e3b-2d3b-46f6-a69f-2bf6740f4f08 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.310380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] MFA Rules not processed for user `83e06ab7cd924e07a79efaafa078a174`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:35.311937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.341297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 863/1729] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:35 2022] PUT /identity/v3/projects/ccc937f4c3b64b4494939198c3b14c47/users/33d51aaf5c4a4a9ab9412a11f103466a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:35.348532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.349037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.349626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.392530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.485023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] MFA Rules not processed for user `33d51aaf5c4a4a9ab9412a11f103466a`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:35.493633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.530728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 867/1730] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.727114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 864/1731] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.740396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.742518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.912548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.916020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.958801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b769358-88e5-42a6-87f2-582cc63d9c7f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.963527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.964586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.987857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.998490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.019735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.022084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.083550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 868/1732] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:36.087959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5086d-4a8e-48d8-ab57-e1184d0af717 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.102342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.103243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.103532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.103812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] PATH_INFO: `/v3/projects/98f3b8e5987e4daebdcfdd6311032a67` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.105097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f3b8e5987e4daebdcfdd6311032a67)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.107152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.315096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f3b8e5987e4daebdcfdd6311032a67', 'request_id': 'req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.336788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f3b8e5987e4daebdcfdd6311032a67', 'request_id': 'req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.356551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.358075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.391372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce26c9-11e3-4a7f-91e2-8ab28f18d145 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.516105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f3b8e5987e4daebdcfdd6311032a67', 'request_id': 'req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.539235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] The token cache is being invalidate because project 98f3b8e5987e4daebdcfdd6311032a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:36.539544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.544257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf19bedd-d5f8-4c6b-9f02-a8679e847ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.550046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 865/1733] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:35 2022] DELETE /identity/v3/projects/98f3b8e5987e4daebdcfdd6311032a67 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:36.551144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936281a-a1d3-47d3-839d-947151de2778 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.551440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936281a-a1d3-47d3-839d-947151de2778 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.551715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936281a-a1d3-47d3-839d-947151de2778 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.565816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 866/1734] 10.222.0.30 () {54 vars in 912 bytes} [Tue Jul 12 14:39:36 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:36.567027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e20778-7e88-486c-8db5-f8e71915ecb1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.567321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e20778-7e88-486c-8db5-f8e71915ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.567602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e20778-7e88-486c-8db5-f8e71915ecb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.569629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.570298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.570584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.570863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.574987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.628138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-11e20778-7e88-486c-8db5-f8e71915ecb1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:36.629414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-11e20778-7e88-486c-8db5-f8e71915ecb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.745830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.747531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcf5f6-6f17-4dad-85eb-76189cea2716 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.775528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 869/1735] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:36.777142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.784799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.784799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.796537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 867/1736] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:36.804945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.805435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.805858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.825222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:36.825222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.850229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:36.854616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.878576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 868/1737] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:36.882541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.894099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb0823-d869-4f64-909d-636f7292650f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.910838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.911783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.912220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.912643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.913581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:36.914039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.916668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.918049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:36.941559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 869/1738] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:36.942782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.965221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4edc7-5428-4db5-b230-6d5a92ef6401 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.984820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 870/1739] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:36.987807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.989558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.000214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.001011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.001427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.005885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.006950 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.015596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.177057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 870/1740] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:36 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:37.178303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.191379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.282452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82016e-5711-4601-9eae-c960d4c9a8eb None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.322035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2af3f-342d-4eee-ae48-f70fc2a53580 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.327107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.327986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.328107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.328338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.346955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.348133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.364189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 871/1741] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:37.365043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.487660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e48b-1a22-484c-b5fb-704995c41a9d service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.514258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.518426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.518426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.518426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.519272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.528363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.529092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.529558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.530307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.531456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.533586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.643860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 872/1742] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:37.648858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.680280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c073c-0a2e-4fa1-82eb-7958f5e47406 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.686220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.687123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.687515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.687892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.688724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.690481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.701745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.703062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.706182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 873/1743] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:37.708412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.724570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f008459-4d18-456f-a5de-6206f38846dc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.731014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.732901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.733407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.733973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.735539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.737549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fd76a-bd57-4067-b665-d3d350e26337 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.743138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 871/1744] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:37.754625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 874/1745] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:37.758865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.784019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81560f2e-bcb3-4cc5-b250-683fe0755ac0 service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.796387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.797727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.798212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.801349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] PATH_INFO: `/v3/projects/eb3256d1556d48dea3804ffaf68a23c7/users/4965ad2c40a84d2ea622123efedc7e41/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.824339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3256d1556d48dea3804ffaf68a23c7, user_id=4965ad2c40a84d2ea622123efedc7e41, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.824777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.910712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 872/1746] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:37 2022] PUT /identity/v3/projects/eb3256d1556d48dea3804ffaf68a23c7/users/4965ad2c40a84d2ea622123efedc7e41/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:37.921431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.921925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.922175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.950388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] MFA Rules not processed for user `4965ad2c40a84d2ea622123efedc7e41`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:37.952170 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:38.074007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.074494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 875/1747] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:38.231938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:38.250286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a6474-1eca-4cf6-8362-5d3c4c6ca447 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.276817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.276817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.276817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.276817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.277097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.278618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030f4d3-4b59-4057-93b1-548823252eda admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.443019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.454004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 873/1748] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:38 2022] POST /identity/v3/projects => generated 385 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:38.465769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe21bf0-8b02-47b0-87fb-40344b7e060f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.562686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.563061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.563501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.584456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.585443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.588471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.588471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.588471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.599298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.616417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] MFA Rules not processed for user `a760d437b597441fa9a4595995c5bc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:38.622263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:38.711627 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 876/1749] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:38 2022] POST /identity/v3/users => generated 299 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:38.715281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.736542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e07e7-b9a6-4829-a01e-210a60c67be3 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.745041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 874/1750] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:38.756720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.760447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.760786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.761078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.761867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.763758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.793202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 877/1751] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:38.795969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.816764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ea4b-c3be-4422-8c38-ca8b5b457d14 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.824425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.825394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.826354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.826662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] PATH_INFO: `/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637/users/429d3365eafd4bd19241354cd2494c05/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.844631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:38.851141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd4136c567a4944a7f6e7b0b0ec3637, user_id=429d3365eafd4bd19241354cd2494c05, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.855251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.945022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 875/1752] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:38 2022] PUT /identity/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637/users/429d3365eafd4bd19241354cd2494c05/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:38.945301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.971115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171b5-6ead-48ec-b5c7-00b853aafe42 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.069565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.070373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.070660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.071101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.071911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.074191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e56088-5531-4c9f-8dd5-b7957b2377e5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.088417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 878/1753] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:39.094040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.102376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020db35-2a26-4406-9b69-09ec44e65c30 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.109583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.110688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.111034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.111319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] PATH_INFO: `/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637/users/429d3365eafd4bd19241354cd2494c05/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.113408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd4136c567a4944a7f6e7b0b0ec3637, user_id=429d3365eafd4bd19241354cd2494c05, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.116333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.128575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 876/1754] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:39 2022] PUT /identity/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637/users/429d3365eafd4bd19241354cd2494c05/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:39.140982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.141358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.141649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.194680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] MFA Rules not processed for user `429d3365eafd4bd19241354cd2494c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:39.196543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.333890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 879/1755] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:39.340821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.343233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.518797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c777-7888-4a3c-a3b2-cf08df8ee23e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.530973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.531665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.531960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.532242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.552756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.560349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d78283-a0b3-4e31-a753-1a2d4636caff service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.577343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 877/1756] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:39.657918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.788449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f2d3-92b3-4e0b-8e1e-497a66abca8a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.795557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.796987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.797375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.797855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] PATH_INFO: `/v3/users/4965ad2c40a84d2ea622123efedc7e41` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.799691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4965ad2c40a84d2ea622123efedc7e41)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.802184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.814457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:40.007615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:40.211186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4965ad2c40a84d2ea622123efedc7e41'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.213189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4965ad2c40a84d2ea622123efedc7e41'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.213645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4965ad2c40a84d2ea622123efedc7e41'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.234923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4965ad2c40a84d2ea622123efedc7e41'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.276255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4965ad2c40a84d2ea622123efedc7e41'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.361203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 880/1757] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:39 2022] DELETE /identity/v3/users/4965ad2c40a84d2ea622123efedc7e41 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:40.364777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.364777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.364937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.390810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:40.397822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.474582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 878/1758] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:40.484886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.487029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.651115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48656da6-c8fc-4bca-b8b2-4e689cb909b6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:40.672346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:40.674641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.676099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.677713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.705624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:40.708697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.751755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 881/1759] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:41.313707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:41.352291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.353832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:41.385615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feed955-4cb0-43ee-82d9-7aec796f3ec6 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.412266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.413237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.413718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.414164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] PATH_INFO: `/v3/projects/eb3256d1556d48dea3804ffaf68a23c7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.415671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3256d1556d48dea3804ffaf68a23c7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.417882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.670859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3256d1556d48dea3804ffaf68a23c7', 'request_id': 'req-14d0fb20-840f-40e0-a8fa-f7a151273aa0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:41.694371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3256d1556d48dea3804ffaf68a23c7', 'request_id': 'req-14d0fb20-840f-40e0-a8fa-f7a151273aa0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:41.819637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3256d1556d48dea3804ffaf68a23c7', 'request_id': 'req-14d0fb20-840f-40e0-a8fa-f7a151273aa0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:41.848409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] The token cache is being invalidate because project eb3256d1556d48dea3804ffaf68a23c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:41.848940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:41.851665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-14d0fb20-840f-40e0-a8fa-f7a151273aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:41.860768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 879/1760] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:41 2022] DELETE /identity/v3/projects/eb3256d1556d48dea3804ffaf68a23c7 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:41.872554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.895517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170e5c8-c730-4c07-abf0-8f96b330715c service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:42.092288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:42.097308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.097730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.098029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] PATH_INFO: `/v3/users/429d3365eafd4bd19241354cd2494c05` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.103777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] RBAC: Authorizing `identity:delete_user(user_id=429d3365eafd4bd19241354cd2494c05)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:42.107681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:42.451661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429d3365eafd4bd19241354cd2494c05'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.455077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429d3365eafd4bd19241354cd2494c05'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.460025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429d3365eafd4bd19241354cd2494c05'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.476218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429d3365eafd4bd19241354cd2494c05'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.515323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429d3365eafd4bd19241354cd2494c05'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.610800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:42.621648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 882/1761] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:41 2022] DELETE /identity/v3/users/429d3365eafd4bd19241354cd2494c05 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:42.739201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.740087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.740537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.878175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:42.884600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:43.029572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:43.036591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 880/1762] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:43.518384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.519037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:43.730377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b946b6-d59a-4212-9e40-5420e95b826c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.745911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.746924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.747970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.748716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.784982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.787134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08613a6-8d25-411e-b5e0-3acd2c0affbe service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.841351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 883/1763] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:39:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:39:44.253254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:44.321377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.325032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.455418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90941ba3-bc62-40ff-b94c-c81ca9a8eb5c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.461031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.462436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.463028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.463741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] PATH_INFO: `/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.465583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd4136c567a4944a7f6e7b0b0ec3637)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.467907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.665289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd4136c567a4944a7f6e7b0b0ec3637', 'request_id': 'req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:44.684407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd4136c567a4944a7f6e7b0b0ec3637', 'request_id': 'req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:44.761976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd4136c567a4944a7f6e7b0b0ec3637', 'request_id': 'req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:44.784087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] The token cache is being invalidate because project 2dd4136c567a4944a7f6e7b0b0ec3637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:44.785061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:44.785793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e15fb8c1-ae85-4995-88ae-95c2a4f092b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:44.788048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 881/1764] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:44 2022] DELETE /identity/v3/projects/2dd4136c567a4944a7f6e7b0b0ec3637 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:44.792979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.793536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.793629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.822186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:44.823489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.895045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 884/1765] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:44.902931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.904265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.904265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.931836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:44.933258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.945331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 882/1766] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:44.946966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.962172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f551e-af71-4e32-9df8-3658f6f1dd4d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.969787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.970867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.971338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.971837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.972790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:44.973324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.975282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.989396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 885/1767] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:44.993884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.003925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17065e36-2bf9-4e3a-8aaa-060a6b836f99 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.010040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.010717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.011776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.011776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.012013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.014186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.117187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 883/1768] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:45.121386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.133605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd769dd-99c3-4d61-af1a-a850ddd5e9e5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.189888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.190595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.190901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.191178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.193062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.195071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.258184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 886/1769] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:45 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:45.262141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.273098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2c82-57b7-4170-b18f-081f6f17f1ec admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.278011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.279287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.279589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.279870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.280597 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.282223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.292606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 884/1770] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:45.296507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.306810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f344a6-57e2-41f3-a888-bf4e59b03b26 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.311421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.312090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.312391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.312654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.313413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.315038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ed802-cf90-48b3-aac7-5e1c9ee99c68 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.326312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 887/1771] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:45.330454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.339180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:45.342465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084f26-2b52-4b8a-a7d4-35997069df42 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.347784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.348706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.348988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.349296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] PATH_INFO: `/v3/projects/79eb54378c084f74afef00a94ea2834d/users/e321d39c76ca480c9f9bd646c17e04da/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.364343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eb54378c084f74afef00a94ea2834d, user_id=e321d39c76ca480c9f9bd646c17e04da, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.367059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.441854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 885/1772] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:45 2022] PUT /identity/v3/projects/79eb54378c084f74afef00a94ea2834d/users/e321d39c76ca480c9f9bd646c17e04da/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:45.453786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.454106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.454387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.482633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] MFA Rules not processed for user `e321d39c76ca480c9f9bd646c17e04da`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:45.483849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:45.553214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 888/1773] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:45.558008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.584518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:45.621492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2894f-adc8-4aa1-8a50-4d86eb7257ff admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.627228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.627887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.628171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.628476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.629229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.631131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.724954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 886/1774] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:45 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:45.725897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.739055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b669b-75f0-42d4-9189-9055c14becef None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.822075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.824259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.824552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.824828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.826191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.828812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.831748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:45.960706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 889/1775] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:45 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:45.962095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:45.975119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e968438-727c-4bdc-bd91-ddad46195a74 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.980665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.981448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.982258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.982536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.983265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.984858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.996005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 887/1776] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:46.001743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.011794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0533b0-9a53-4b62-b1e2-da1bb4879983 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.019961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.019961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.020154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.020433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] PATH_INFO: `/v3/projects/20e96de827a64814bd4df004f33e0ebe/users/a53001a6c92f477296d7c32043c0b388/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.035843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e96de827a64814bd4df004f33e0ebe, user_id=a53001a6c92f477296d7c32043c0b388, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.038823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.115381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 890/1777] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:46 2022] PUT /identity/v3/projects/20e96de827a64814bd4df004f33e0ebe/users/a53001a6c92f477296d7c32043c0b388/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:46.119445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.130211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b7b7-0aa7-4f0d-8854-42618662a81e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.184379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.185043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.185358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.185635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.187007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.188592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366fc7a-eebe-4cec-a3d1-d76b1c3a9353 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.198270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 888/1778] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:46.202309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.212240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191c0e-1504-4479-b765-76db2100a329 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.216738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.217644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.217948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.218196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] PATH_INFO: `/v3/projects/20e96de827a64814bd4df004f33e0ebe/users/a53001a6c92f477296d7c32043c0b388/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.220142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e96de827a64814bd4df004f33e0ebe, user_id=a53001a6c92f477296d7c32043c0b388, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.222789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.233552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 891/1779] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:46 2022] PUT /identity/v3/projects/20e96de827a64814bd4df004f33e0ebe/users/a53001a6c92f477296d7c32043c0b388/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:46.243583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.243875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.244156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.271457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] MFA Rules not processed for user `a53001a6c92f477296d7c32043c0b388`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:46.272744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:46.340241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 889/1780] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:46.351259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.352961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:46.413395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6d01b-61d3-41e9-89fa-fb237de60ef6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.417135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.417864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.418146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.418425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.428055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:46.430537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.431913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f2fa9-b8b9-47e1-ab6f-666be3f4efd5 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.442589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 892/1781] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:46.633560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.657919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:46.731803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303c33-1dfc-4377-a633-2ccd35fc0fdf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.741197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.742265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.743334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.743334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] PATH_INFO: `/v3/users/e321d39c76ca480c9f9bd646c17e04da` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.744457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e321d39c76ca480c9f9bd646c17e04da)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.746835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.983641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e321d39c76ca480c9f9bd646c17e04da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:47.006943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e321d39c76ca480c9f9bd646c17e04da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:47.009200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e321d39c76ca480c9f9bd646c17e04da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:47.016885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f6ce5-c347-4aa6-a677-60ba2aad6aae None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.017402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f6ce5-c347-4aa6-a677-60ba2aad6aae None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.017692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f6ce5-c347-4aa6-a677-60ba2aad6aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.064769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-010f6ce5-c347-4aa6-a677-60ba2aad6aae None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:47.068659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-010f6ce5-c347-4aa6-a677-60ba2aad6aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:47.068818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e321d39c76ca480c9f9bd646c17e04da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:47.125668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:47.130501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e321d39c76ca480c9f9bd646c17e04da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:47.196145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 893/1782] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Jul 12 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:47.255885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 890/1783] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:46 2022] DELETE /identity/v3/users/e321d39c76ca480c9f9bd646c17e04da => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:47.264599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.264903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.265371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.319164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:47.322970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:47.402083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:47.403580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:47.690569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 894/1784] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:47.690569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:47.690569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:47.691792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603505-1b94-4aa2-82c7-b94fa28b20e2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:47.691959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e89f2-32e9-459e-9558-e79b06302b22 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:47.692122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:47.692122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.692122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.692122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.694000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:47.694000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.694000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.694000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.694582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:47.694582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:47.694727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:47.694727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:47.694859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 891/1785] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:47.694932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 895/1786] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:47.707649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:47.722574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef70c5-28b0-4606-b419-c2b1eaf13778 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:47.734816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:47.734816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.734816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.734816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.745903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:47.747894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a60e-264d-49ab-85c4-035f939de921 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:47.768390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 892/1787] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:48.272096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:48.274189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:48.305041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf30aa-e040-4c1e-a24a-ef6f7fc84f1c service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:48.318144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:48.319523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:48.319523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:48.319751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] PATH_INFO: `/v3/users/55b13cfb5c3541fbbf631fd5621518c3` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:48.322448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b13cfb5c3541fbbf631fd5621518c3)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:48.324658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:48.602549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b13cfb5c3541fbbf631fd5621518c3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:48.603991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b13cfb5c3541fbbf631fd5621518c3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:48.605111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b13cfb5c3541fbbf631fd5621518c3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:48.636045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b13cfb5c3541fbbf631fd5621518c3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:48.683574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b13cfb5c3541fbbf631fd5621518c3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:48.771198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 896/1788] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:48 2022] DELETE /identity/v3/users/55b13cfb5c3541fbbf631fd5621518c3 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:48.773719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:48.774235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:48.774402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:48.806862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:48.808600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:48.846602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:48.952025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 893/1789] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:49.010254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:49.016771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:49.147893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:49.208056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c73b9-ed7f-4aa5-9372-5e09019c6553 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:49.222953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:49.223645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:49.224055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:49.224234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:49.251810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:49.253993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9e9c5-7438-4d6f-a950-04124bf597f2 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:49.309857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 897/1790] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:50.294934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:50.297724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:50.330176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327093a-5cdd-4b4c-b14c-c901f10e02fe None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:50.352760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:50.358756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:50.359070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:50.360799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] PATH_INFO: `/v3/projects/79eb54378c084f74afef00a94ea2834d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:50.367375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=79eb54378c084f74afef00a94ea2834d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:50.392159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84920-3ed5-4d3c-a32b-5fd12d9003d9 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:50.392738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84920-3ed5-4d3c-a32b-5fd12d9003d9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:50.392738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84920-3ed5-4d3c-a32b-5fd12d9003d9 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:50.393642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:50.394836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 898/1791] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:39:50.413401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:50.414084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:50.414604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:50.467602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:50.468474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:50.640744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 899/1792] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:50.643655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79eb54378c084f74afef00a94ea2834d', 'request_id': 'req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:50.669315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79eb54378c084f74afef00a94ea2834d', 'request_id': 'req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:50.756350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79eb54378c084f74afef00a94ea2834d', 'request_id': 'req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:50.780723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] The token cache is being invalidate because project 79eb54378c084f74afef00a94ea2834d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:50.781251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:50.781737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:50.787363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 894/1793] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:50 2022] DELETE /identity/v3/projects/79eb54378c084f74afef00a94ea2834d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:50.788744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:50.804958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e52b97-c421-4e56-91e4-59e25443c70e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:50.990556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:50.993264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:50.997402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:50.997402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] PATH_INFO: `/v3/users/a53001a6c92f477296d7c32043c0b388` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:50.997402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53001a6c92f477296d7c32043c0b388)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:50.999943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:51.059671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:51.061972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:51.228849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60b3f-f47b-450f-9e7a-ef93e975d7bb admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:51.253315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:51.254112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:51.254395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:51.262828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:51.267073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:51.300986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:51.302403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:51.345974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53001a6c92f477296d7c32043c0b388'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.349578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53001a6c92f477296d7c32043c0b388'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.356900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53001a6c92f477296d7c32043c0b388'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.358117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 895/1794] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:51.377852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53001a6c92f477296d7c32043c0b388'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.417271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53001a6c92f477296d7c32043c0b388'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.417271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:51.500368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 900/1795] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:50 2022] DELETE /identity/v3/users/a53001a6c92f477296d7c32043c0b388 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:52.603928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:52.605494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:52.634830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9035-0343-4f70-b2c1-f00431543dda service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:52.716331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:52.724160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:52.749333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:52.808070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:52.809224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:52.810112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:52.810112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] PATH_INFO: `/v3/projects/b8dac4a3c40747b1970dc9b73b237a56` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:52.813972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8dac4a3c40747b1970dc9b73b237a56)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:52.816460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:52.896679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b91aa-7582-4735-bd05-f0100d399425 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:52.906679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:52.909444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:52.910458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:52.911399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] PATH_INFO: `/v3/users/2cd5465169634b85bbd56e49fd738f5e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:52.914091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd5465169634b85bbd56e49fd738f5e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:52.928477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:52.978914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8dac4a3c40747b1970dc9b73b237a56', 'request_id': 'req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.995013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8dac4a3c40747b1970dc9b73b237a56', 'request_id': 'req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.124188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8dac4a3c40747b1970dc9b73b237a56', 'request_id': 'req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.141625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] The token cache is being invalidate because project b8dac4a3c40747b1970dc9b73b237a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:53.143890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.145242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.148804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 896/1796] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:52 2022] DELETE /identity/v3/projects/b8dac4a3c40747b1970dc9b73b237a56 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:53.152885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.155465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.188299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd5465169634b85bbd56e49fd738f5e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.195176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd5465169634b85bbd56e49fd738f5e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.200272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd5465169634b85bbd56e49fd738f5e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.228796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd5465169634b85bbd56e49fd738f5e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.280502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10af25ce-7322-41e1-921c-4cf3948b594f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd5465169634b85bbd56e49fd738f5e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.367108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4a0e3-5c10-4a43-adb9-3a843e93aeed admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.384303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.385333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.387743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.388097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] PATH_INFO: `/v3/users/a660528a466343aeb54a551bc755ce8d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.394039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a660528a466343aeb54a551bc755ce8d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.397447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.407796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 901/1797] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:52 2022] DELETE /identity/v3/users/2cd5465169634b85bbd56e49fd738f5e => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:53.409480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e3d7-6078-4b76-b0ea-18ec80eda457 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.413993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e3d7-6078-4b76-b0ea-18ec80eda457 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.414335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e3d7-6078-4b76-b0ea-18ec80eda457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.487841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0210e3d7-6078-4b76-b0ea-18ec80eda457 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:53.489208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0210e3d7-6078-4b76-b0ea-18ec80eda457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.645271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 902/1798] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:53.648148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.648448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.648740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.650834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 903/1799] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:39:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:39:53.690153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a660528a466343aeb54a551bc755ce8d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.691611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a660528a466343aeb54a551bc755ce8d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.692996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a660528a466343aeb54a551bc755ce8d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.717748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.718436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a660528a466343aeb54a551bc755ce8d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.721457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.762278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:53.767241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e0694f43-2592-4d16-a73f-04f9cdf9a2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a660528a466343aeb54a551bc755ce8d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.832810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 897/1800] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:53 2022] DELETE /identity/v3/users/a660528a466343aeb54a551bc755ce8d => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:53.872784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188909c-1d62-42c7-800b-eee946700030 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.874178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188909c-1d62-42c7-800b-eee946700030 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.875282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188909c-1d62-42c7-800b-eee946700030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.898367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c995-f90d-4578-be02-b591dfcdf26d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.948557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-5188909c-1d62-42c7-800b-eee946700030 None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:53.950457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5188909c-1d62-42c7-800b-eee946700030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:54.028359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.029234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.029599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.029887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.037365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:54.129369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 898/1801] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:54.193800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.195638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.236005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 904/1802] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:54.670665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:54.873287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188909c-1d62-42c7-800b-eee946700030 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.876970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5188909c-1d62-42c7-800b-eee946700030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:54.908826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188909c-1d62-42c7-800b-eee946700030 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.918945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.949722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.952840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.953823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] PATH_INFO: `/v3/projects/20e96de827a64814bd4df004f33e0ebe` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.958536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e96de827a64814bd4df004f33e0ebe)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.967945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.233506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e96de827a64814bd4df004f33e0ebe', 'request_id': 'req-a01736f5-83be-437d-8c85-199d99b38efc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.254009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:55.258212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.262009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e96de827a64814bd4df004f33e0ebe', 'request_id': 'req-a01736f5-83be-437d-8c85-199d99b38efc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.302684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eb0eb-1c14-4343-a665-350f7e213c17 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:55.355692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e96de827a64814bd4df004f33e0ebe', 'request_id': 'req-a01736f5-83be-437d-8c85-199d99b38efc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.381788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] The token cache is being invalidate because project 20e96de827a64814bd4df004f33e0ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:55.382761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.383813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a01736f5-83be-437d-8c85-199d99b38efc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.386265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 899/1803] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:54 2022] DELETE /identity/v3/projects/20e96de827a64814bd4df004f33e0ebe => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:55.409606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.410146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.410443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.457197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:55.458375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.589924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:55.589924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.590275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.590393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.601769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.615829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 900/1804] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:55.615829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:55.619150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.641508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:55.649132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30214d04-24ea-40b3-bc18-e6b1ce144aa7 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.650465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f217c-19a0-440c-ade8-8bd1b62e3564 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:55.659938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:55.663820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.664114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.664792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] PATH_INFO: `/v3/projects/bbd513ca99a9472f8309b7c1310f7f87` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.667072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd513ca99a9472f8309b7c1310f7f87)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:55.670716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.682897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 905/1805] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:55.684393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.688039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.688039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.712122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:55.729256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:55.730542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.765317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 906/1806] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:55.776400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:55.796415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caff96-f6c6-4156-be7d-502c63fd0772 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:55.812077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd513ca99a9472f8309b7c1310f7f87', 'request_id': 'req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.819233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd513ca99a9472f8309b7c1310f7f87', 'request_id': 'req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.906658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd513ca99a9472f8309b7c1310f7f87', 'request_id': 'req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.914272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:55.915203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.915678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.916125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.918312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:55.918312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:55.918610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:55.919705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.927374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] The token cache is being invalidate because project bbd513ca99a9472f8309b7c1310f7f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:55.927804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.928721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-27fea4c7-46f1-4de7-a5ce-58798a2eabf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:55.930484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 901/1807] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:55 2022] DELETE /identity/v3/projects/bbd513ca99a9472f8309b7c1310f7f87 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:55.936764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066d0e-9b5a-48af-9594-c2f7df6057e2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.937347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066d0e-9b5a-48af-9594-c2f7df6057e2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.937790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c066d0e-9b5a-48af-9594-c2f7df6057e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.938118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 907/1808] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:55.943034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:55.945105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.971160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7c066d0e-9b5a-48af-9594-c2f7df6057e2 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:55.971160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7c066d0e-9b5a-48af-9594-c2f7df6057e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.971445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae920a3-ed4c-4a83-ab82-7bda1ee5ff6b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:55.979405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:55.980664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.981133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.982494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.982757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 902/1809] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:55.983460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:55.985962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.994142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.994142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.994142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.042799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:56.044782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:56.077687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 903/1810] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:56.082141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.104395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76cab2-df40-41ff-8d10-a8a0b5e79abf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.119411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.120242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.121790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.122179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.125574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:56.125908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.129889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.153107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 904/1811] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:39:56.158082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.160036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:56.176244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 908/1812] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:55 2022] POST /identity/v3/projects => generated 409 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:56.181455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.189488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13c80-707c-4aa3-8831-28b74133c1fe admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.402419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.403323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.403607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.403926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] PATH_INFO: `/v3/projects/5cd73adf4ec049ceb284156dd16ccf04` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.406156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd73adf4ec049ceb284156dd16ccf04)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.408144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.412767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3996e-3579-4b18-9fd7-b33b80d742f8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.416413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.417286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.417645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.418059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.418885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.421055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.553122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 909/1813] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:56 2022] POST /identity/v3/projects => generated 367 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:56.554590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.583638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd73adf4ec049ceb284156dd16ccf04', 'request_id': 'req-8a4166dd-900c-4e87-928b-e813714811f1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:56.625550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd73adf4ec049ceb284156dd16ccf04', 'request_id': 'req-8a4166dd-900c-4e87-928b-e813714811f1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:56.695903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:56.714361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747d04a-24f1-4cc8-a872-d60fddaeede0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.722143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.722870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.723158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.723843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.724587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.726704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.744952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd73adf4ec049ceb284156dd16ccf04', 'request_id': 'req-8a4166dd-900c-4e87-928b-e813714811f1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:56.764428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] The token cache is being invalidate because project 5cd73adf4ec049ceb284156dd16ccf04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:56.764932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:56.771270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:56.773455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 905/1814] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:56 2022] DELETE /identity/v3/projects/5cd73adf4ec049ceb284156dd16ccf04 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:56.774339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.776445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:56.855028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 910/1815] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:56 2022] POST /identity/v3/users => generated 305 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:56.855945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.859943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:56.899425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be775453-ac47-4018-adf5-ebda6baca7d6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.908408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.909348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.909647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.913194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] PATH_INFO: `/v3/users/33d51aaf5c4a4a9ab9412a11f103466a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.913194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d51aaf5c4a4a9ab9412a11f103466a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.915936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.930964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4166dd-900c-4e87-928b-e813714811f1 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.949039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.949673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.952427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.952664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:56.956765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:56.965753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:56.978723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:57.135395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 906/1816] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:56 2022] POST /identity/v3/users => generated 284 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:57.135942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.138026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:57.174951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d51aaf5c4a4a9ab9412a11f103466a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:57.175628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d51aaf5c4a4a9ab9412a11f103466a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:57.176387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d51aaf5c4a4a9ab9412a11f103466a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:57.180984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81406-965e-4e7c-95af-b00ba166e2f4 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.204635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.209544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.209544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.209544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.209544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.209544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.228719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d51aaf5c4a4a9ab9412a11f103466a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:57.230837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 907/1817] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:57.231927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.234464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:57.250638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc14ad-3bb7-4914-8ee4-3eeb7a25ae8c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.256559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d51aaf5c4a4a9ab9412a11f103466a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:57.262560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.264365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.264654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.266963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.266963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.269210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.285781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 908/1818] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:57.289872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.289872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:57.325182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf84da1-658a-4379-bde4-ea209a221943 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.327810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.328503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.328793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.329185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.330025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.331710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.347267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 911/1819] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:56 2022] DELETE /identity/v3/users/33d51aaf5c4a4a9ab9412a11f103466a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:57.347267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.365224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 909/1820] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:57.378353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.406454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903bf2b-2eac-49ed-8408-f26df84d2fc0 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.509769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6cfcd-420b-4ca8-b077-66f3549e8774 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.514635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.515608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.515896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.516227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.517384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.519265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.528802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.537083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 912/1821] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:57.541791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.542220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.542920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.543234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] PATH_INFO: `/v3/projects/98d908d54689431e8c972da268238f6d/users/2104e109cc514e5997aed1401bdb3af7/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.560011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f35ae2-8b00-4f5b-b714-0a53e5cc1f21 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.566721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.568775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.569473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.569988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] PATH_INFO: `/v3/projects/bd004b5c44904817a7a42a00700b6856/users/5a8662955acd48ebbce71914a3b5c163/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.578901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d908d54689431e8c972da268238f6d, user_id=2104e109cc514e5997aed1401bdb3af7, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.583379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856db416-148a-4a75-85f7-96d3d813a594 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.593716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd004b5c44904817a7a42a00700b6856, user_id=5a8662955acd48ebbce71914a3b5c163, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.597006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc203a-d169-4972-ae1d-1dd92ffdbfb2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.674799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 910/1822] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:57 2022] PUT /identity/v3/projects/98d908d54689431e8c972da268238f6d/users/2104e109cc514e5997aed1401bdb3af7/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:57.686428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.686811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.687001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.697965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 913/1823] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:57 2022] PUT /identity/v3/projects/bd004b5c44904817a7a42a00700b6856/users/5a8662955acd48ebbce71914a3b5c163/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:57.713318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.714080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.714620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.757866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] MFA Rules not processed for user `2104e109cc514e5997aed1401bdb3af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:57.759609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:57.770744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] MFA Rules not processed for user `5a8662955acd48ebbce71914a3b5c163`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:57.771987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:57.811031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:57.896913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 911/1824] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:57.898014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.926575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 914/1825] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:57.934515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.077779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a6642-bdd9-4a2c-a49a-687a861de65d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.099805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.101216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.101707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.103535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.113191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.115659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.139954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e407c-278c-4b3b-8a73-b48c31804c88 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.146348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.147134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.147674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.148047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.148682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.158102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.269762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 912/1826] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:57 2022] POST /identity/v3/projects => generated 407 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:58.275460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 915/1827] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:57 2022] POST /identity/v3/projects => generated 367 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:58.276290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.279732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.489990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9593c5-9e0a-4411-82c4-a65f7c7b0d76 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.495384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.496094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.496671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.496826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.498548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.500890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.507035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ae006-314a-490a-9969-79a4d410701b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.514679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.515534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.515892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.516305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.517188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.521356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.527036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:58.590549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 913/1828] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:58 2022] POST /identity/v3/users => generated 290 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:58.597719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.616270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c0bb8-b1ec-4df6-8e2c-e55b1d49da46 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.626550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.626550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.626550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.626550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.627060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.628923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.648857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 916/1829] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:58 2022] POST /identity/v3/users => generated 310 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:58.657114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.664382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 914/1830] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:58.668876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.672892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815bb7e-a422-433f-95e8-9cbbc3f37087 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.678904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.682805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.682805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.682805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.683291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.686411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.705777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 917/1831] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:58.706737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.717556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9344587-eb4d-4ce9-8699-add80c3b957f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.738583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.739243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.739673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.739847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] PATH_INFO: `/v3/projects/97a2642bcf6547758ad69702bb8168e6/users/b8366bbdab1a4ed8b80b608e05cb261d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.743186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbcd87-bd8a-4a82-9e2b-780e75ea1a1d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.750042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.751412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.751894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.752530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] PATH_INFO: `/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0/users/8caca919fc8147d29f18c91cef54d447/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.760659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a2642bcf6547758ad69702bb8168e6, user_id=b8366bbdab1a4ed8b80b608e05cb261d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.764790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.774347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3563a9f411342eea5fbfe6e5b0396a0, user_id=8caca919fc8147d29f18c91cef54d447, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.777968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:58.847701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 915/1832] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:58 2022] PUT /identity/v3/projects/97a2642bcf6547758ad69702bb8168e6/users/b8366bbdab1a4ed8b80b608e05cb261d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:58.851688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.870878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 918/1833] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:58 2022] PUT /identity/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0/users/8caca919fc8147d29f18c91cef54d447/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:58.876650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:58.957631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceca3ce-13f5-4ef0-8949-cccd38241461 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.986132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fedb1c-6f01-4b9c-88ab-6c1ec9c82d83 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:58.991736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:58.992554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:58.992936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:58.993431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:58.994409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:58.996383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.019899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 919/1834] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:59.024246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.035429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f7a7-0ab2-4c4c-8f86-8d5d80f56521 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.036940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.037753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.038142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.038489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.039372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.040505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.041503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.041697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.042032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.042314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] PATH_INFO: `/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0/users/8caca919fc8147d29f18c91cef54d447/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.044298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3563a9f411342eea5fbfe6e5b0396a0, user_id=8caca919fc8147d29f18c91cef54d447, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.047243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08757093-6e78-4e53-aaa5-fc635457c3d8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.051848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 916/1835] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:59.055854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.058086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 920/1836] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:59 2022] PUT /identity/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0/users/8caca919fc8147d29f18c91cef54d447/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:59.065818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26381d42-04f8-45df-afad-942acaf4a26a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.068239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.068536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.068816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.096160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] MFA Rules not processed for user `8caca919fc8147d29f18c91cef54d447`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:59.097568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.123618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.124510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.124797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.125073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] PATH_INFO: `/v3/projects/97a2642bcf6547758ad69702bb8168e6/users/b8366bbdab1a4ed8b80b608e05cb261d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.127112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a2642bcf6547758ad69702bb8168e6, user_id=b8366bbdab1a4ed8b80b608e05cb261d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.129896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35de35-e4ef-4b16-860e-0ea17fad10ab admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.146837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 917/1837] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:59 2022] PUT /identity/v3/projects/97a2642bcf6547758ad69702bb8168e6/users/b8366bbdab1a4ed8b80b608e05cb261d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:59.153872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.154173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.154621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.177095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 921/1838] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:59.187837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] MFA Rules not processed for user `b8366bbdab1a4ed8b80b608e05cb261d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:59.188900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.189074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.190329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.254448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfc0c-f0f6-4ac3-aad6-718eab19b25c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.269743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 918/1839] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:59.281666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.283074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.313180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.313845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.314127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.314400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.347706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488ce1-d57b-48f2-b29e-0e610017b42d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.352023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.352683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.352971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.353312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.363452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.364979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.375730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 919/1840] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:59.378198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.383211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.393272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 922/1841] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:59.436652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.441668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:59.442058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.519912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c09e8-396c-4814-87cd-ce646a4d0bdc service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.524549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.525432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.525720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.526001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] PATH_INFO: `/v3/projects/ccc937f4c3b64b4494939198c3b14c47` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.527194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc937f4c3b64b4494939198c3b14c47)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.529243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.541199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.550297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db489aa-da37-4ac3-9349-70002fdf9d64 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.554083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.554729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.555010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.555283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.611063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.612493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb73d5a-acb7-40ae-9a46-ec116dd24bca service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.621707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 923/1842] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:59.672170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc937f4c3b64b4494939198c3b14c47', 'request_id': 'req-76679864-ad20-42a7-b531-36d68562c7cc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:59.687177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc937f4c3b64b4494939198c3b14c47', 'request_id': 'req-76679864-ad20-42a7-b531-36d68562c7cc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:59.806426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc937f4c3b64b4494939198c3b14c47', 'request_id': 'req-76679864-ad20-42a7-b531-36d68562c7cc'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:59.819024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] The token cache is being invalidate because project ccc937f4c3b64b4494939198c3b14c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:59.819313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:59.820081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:59.822155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 920/1843] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:59 2022] DELETE /identity/v3/projects/ccc937f4c3b64b4494939198c3b14c47 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:59.827999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ded1-71ff-447e-a6a9-c7893728444d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.828437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ded1-71ff-447e-a6a9-c7893728444d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.828725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ded1-71ff-447e-a6a9-c7893728444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.847419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1f37ded1-71ff-447e-a6a9-c7893728444d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:59.848645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1f37ded1-71ff-447e-a6a9-c7893728444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.892181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.893702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.921952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 924/1844] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:59.929784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.930144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.930500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.949746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:59.957329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.957450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679864-ad20-42a7-b531-36d68562c7cc admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.957450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.957450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.957450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.957800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.959002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 925/1845] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:59.959191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.963202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.973292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76556d04-e21a-41ec-8d38-bc21f8c9c2a5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.980222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.980222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.980222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.980222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.980609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:59.980609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.982898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.996209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 926/1846] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:40:00.000668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.011293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7424d4-753f-4835-8e76-6b85c6b94daa admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.016460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.017117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.017433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.017706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.018395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.020266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.024473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:00.026726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.028419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.038650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 921/1847] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:00.116966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 927/1848] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:59 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:00.121343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.133740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a65c3-fcbb-4719-be21-1e2560488a77 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.216009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.217883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.218279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.218623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.219628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.222136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.336642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 922/1849] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:00 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:00.337842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.349650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec68bef-aca7-4a54-809d-4553b6f10bb5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.355050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.355740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.357326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:00.360168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.360168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.360168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.360508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.374433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 928/1850] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:00.376442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.386445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726eba-845b-40b3-bb31-3b2600ec41ca admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.391725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.392391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.392679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.392987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.393863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.400299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e1726-285e-4c01-9bf8-7251e8c73feb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.406035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 923/1851] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:00.410128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.419950 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2481e-f34d-4a6d-b1ad-bfbae0252b73 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.424835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.429071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.429071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.429071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] PATH_INFO: `/v3/projects/6ed6f56f8a284fc990994a2467429233/users/2b4fc16a6d6f4c94b8767dbc423c32a4/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.442969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed6f56f8a284fc990994a2467429233, user_id=2b4fc16a6d6f4c94b8767dbc423c32a4, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.445763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.533061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 929/1852] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:00 2022] PUT /identity/v3/projects/6ed6f56f8a284fc990994a2467429233/users/2b4fc16a6d6f4c94b8767dbc423c32a4/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:00.544188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.544477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.544754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.576246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] MFA Rules not processed for user `2b4fc16a6d6f4c94b8767dbc423c32a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:00.577587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:00.643738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 924/1853] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:00.648233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.669290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:00.708728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84818-525b-4132-a1da-3474061881c5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.713367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.714027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.714313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.714592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.715278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.717207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.845455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 930/1854] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:00 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:00.846574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:00.853211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:00.857871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b43f0-fab3-4e4b-a319-1311f98009c9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.946074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.946760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.947048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.947327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.948231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.950224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.080467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 925/1855] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:00 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:01.084635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.095054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a439a7-8051-4a1b-9a04-2085ca95549f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.098046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:01.101115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.101115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.101115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.101115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.101761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.103380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.114382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 931/1856] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:01.118331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.128887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc5551-1342-4356-9130-9d795596a65b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.133480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.134356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.134908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.134908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] PATH_INFO: `/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05/users/1e6f2ad894004ec58ef34fed1a5bb689/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.149186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f23a2b72494e34b7a4c3d2e2ce7b05, user_id=1e6f2ad894004ec58ef34fed1a5bb689, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.151760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.234599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 926/1857] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:01 2022] PUT /identity/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05/users/1e6f2ad894004ec58ef34fed1a5bb689/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:01.239103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.250662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebc58d-a15b-4256-bc1e-bba6c2d598eb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.324642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.325361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.326166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.326449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.327315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.328910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fcf54-07c2-450a-afb9-5b463389d79f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.338200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:01.343161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 932/1858] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:01.346898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.357525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fd21a-a8a4-409b-b2cc-ac9933592aa8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.362481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.363620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.363972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.364369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] PATH_INFO: `/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05/users/1e6f2ad894004ec58ef34fed1a5bb689/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.366507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f23a2b72494e34b7a4c3d2e2ce7b05, user_id=1e6f2ad894004ec58ef34fed1a5bb689, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.369681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.384300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 927/1859] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:01 2022] PUT /identity/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05/users/1e6f2ad894004ec58ef34fed1a5bb689/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:01.396845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.397140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.397454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.404569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:01.424954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] MFA Rules not processed for user `1e6f2ad894004ec58ef34fed1a5bb689`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:01.429764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:01.526942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 933/1860] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:01.537448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.602052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeabea-75ac-47b6-812d-20a24b324837 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.608119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.611018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.611018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.611018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.629112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.631164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.654996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 928/1861] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:01.776531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.786976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f3fe-6835-4acc-9a9d-a24f70ef0615 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.790594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.791278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.791570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.791847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.854828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.856303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8267f8-973d-420f-9d21-523b4aedbbf6 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.866882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 934/1862] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:05.524853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.526329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.537952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:05.605138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1755ab-ad2d-46a8-9b5f-07de4280cb38 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.612254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.613140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.613460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.613741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] PATH_INFO: `/v3/users/5a8662955acd48ebbce71914a3b5c163` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.614948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8662955acd48ebbce71914a3b5c163)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.616927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.843447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8662955acd48ebbce71914a3b5c163'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.844155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8662955acd48ebbce71914a3b5c163'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.844790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8662955acd48ebbce71914a3b5c163'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.851779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8662955acd48ebbce71914a3b5c163'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.863551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8662955acd48ebbce71914a3b5c163'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.939520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 929/1863] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:05 2022] DELETE /identity/v3/users/5a8662955acd48ebbce71914a3b5c163 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:05.944969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85475c-9700-4df1-b098-30809b03563d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.945447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85475c-9700-4df1-b098-30809b03563d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.945750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85475c-9700-4df1-b098-30809b03563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.965247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-de85475c-9700-4df1-b098-30809b03563d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:05.965247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-de85475c-9700-4df1-b098-30809b03563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:06.031466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 935/1864] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:06.039226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:06.040812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:06.101522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2bf4d-7382-437a-9d01-6088454d32de admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:06.105217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:06.106092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:06.106466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:06.106745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:06.116761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:06.118292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:06.128744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 930/1865] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:06.625115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85475c-9700-4df1-b098-30809b03563d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:06.626601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-de85475c-9700-4df1-b098-30809b03563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:06.648446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85475c-9700-4df1-b098-30809b03563d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:06.653135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:06.655491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:06.656119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:06.656405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] PATH_INFO: `/v3/projects/bd004b5c44904817a7a42a00700b6856` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:06.657663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd004b5c44904817a7a42a00700b6856)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:06.659610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:06.687157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:06.789887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd004b5c44904817a7a42a00700b6856', 'request_id': 'req-bec51905-7a8b-401d-940c-ca02aa4bdb6f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:06.796341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd004b5c44904817a7a42a00700b6856', 'request_id': 'req-bec51905-7a8b-401d-940c-ca02aa4bdb6f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:06.857515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd004b5c44904817a7a42a00700b6856', 'request_id': 'req-bec51905-7a8b-401d-940c-ca02aa4bdb6f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:06.874529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] The token cache is being invalidate because project bd004b5c44904817a7a42a00700b6856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:06.874826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:06.875491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:06.881187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 936/1866] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:06 2022] DELETE /identity/v3/projects/bd004b5c44904817a7a42a00700b6856 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:06.881401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:06.891267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5661e-64f1-485a-9b17-73031a6fb16b service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:06.921694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:06.948773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:06.949672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:06.949958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:06.950747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] PATH_INFO: `/v3/users/b8366bbdab1a4ed8b80b608e05cb261d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:06.951951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8366bbdab1a4ed8b80b608e05cb261d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:06.953956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:07.173388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8366bbdab1a4ed8b80b608e05cb261d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:07.174055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8366bbdab1a4ed8b80b608e05cb261d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:07.174664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8366bbdab1a4ed8b80b608e05cb261d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:07.181627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8366bbdab1a4ed8b80b608e05cb261d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:07.193768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0aa64393-e399-4f36-9263-6985dd94c981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8366bbdab1a4ed8b80b608e05cb261d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:07.268631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 931/1867] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:06 2022] DELETE /identity/v3/users/b8366bbdab1a4ed8b80b608e05cb261d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:08.034225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.035706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:08.050348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec51905-7a8b-401d-940c-ca02aa4bdb6f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.141023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.141023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.141023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.141023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] PATH_INFO: `/v3/projects/97a2642bcf6547758ad69702bb8168e6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.142741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a2642bcf6547758ad69702bb8168e6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.144460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.235062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a2642bcf6547758ad69702bb8168e6', 'request_id': 'req-c8dc0da3-2bc0-457b-a66c-c836462d960d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:08.240554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a2642bcf6547758ad69702bb8168e6', 'request_id': 'req-c8dc0da3-2bc0-457b-a66c-c836462d960d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:08.303113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a2642bcf6547758ad69702bb8168e6', 'request_id': 'req-c8dc0da3-2bc0-457b-a66c-c836462d960d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:08.316536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] The token cache is being invalidate because project 97a2642bcf6547758ad69702bb8168e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:08.316833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:08.317512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c8dc0da3-2bc0-457b-a66c-c836462d960d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:08.319649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 937/1868] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:08 2022] DELETE /identity/v3/projects/97a2642bcf6547758ad69702bb8168e6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:08.324619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.325173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.325602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.344667 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:08.345926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:08.410108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 932/1869] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:08.418400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.418689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.419522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.438829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:08.440103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:08.450771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 938/1870] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:08.454736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.464801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf4768-a463-4998-b3ca-7a594c5a548f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.470040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.470696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.470982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.471268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.471992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:08.472303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.473940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.485436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 933/1871] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:08.490021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.499890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1b453-6d4c-4bf6-a220-1dda1ea817e6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.505234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.505884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.506167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.506445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.507193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.509087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.584207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 939/1872] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:08 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:08.588686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.599780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e10b3-147b-4713-8f63-6c7dff96bd18 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.635214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:08.659910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.660689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.660902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.661270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.662095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.664072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.759758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 934/1873] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:08 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:08.761107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.772809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d321-eb23-40b2-934a-b32be2afb615 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.777733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.778986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.779252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.779526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.780260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.781943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.793177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 940/1874] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:08.797418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.808155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7634ff3-c849-48e3-acd7-b53c519f5354 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.812951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.813657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.813948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.814229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.814961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.816558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9199c6-7150-46f1-8901-af37354616f0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.827153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 935/1875] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:08.831212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:08.836433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:08.842478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5e07-5f33-4478-b634-d366e2f92557 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:08.847192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:08.850898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.850898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.850898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] PATH_INFO: `/v3/projects/1d88d7d001a44867bd926bf90fa128d1/users/a7036b95a322471d887f8b1146e1bf3e/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.862893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d88d7d001a44867bd926bf90fa128d1, user_id=a7036b95a322471d887f8b1146e1bf3e, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:08.865561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:08.920572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 941/1876] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:08 2022] PUT /identity/v3/projects/1d88d7d001a44867bd926bf90fa128d1/users/a7036b95a322471d887f8b1146e1bf3e/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:08.931561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:08.931858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:08.932142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:08.971626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] MFA Rules not processed for user `a7036b95a322471d887f8b1146e1bf3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:08.972721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:09.056346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 936/1877] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:09.061602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.169828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80826403-824c-4d19-80b7-5241a50cec89 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.177742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.178454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.178743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.179033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.180224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.183416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.322733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 942/1878] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:09 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:09.324562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.336070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8bb51-ff33-4d8b-b5c0-68c2c4dc0667 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.394831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.395506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.395788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.396066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.396758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.398768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.486789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 937/1879] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:09 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:09.492600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.503930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa30fa-0092-4b03-8008-46506929a6cc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.508553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.509249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.509539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.509819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.510553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.512142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.523186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 943/1880] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:09.527469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.537531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249c320-da54-49f4-8452-bebc7a7a1cd6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.542368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.543290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.543616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.544550 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] PATH_INFO: `/v3/projects/0ebbe533568e48b1bdba51e70eb969eb/users/037bc1ac03554ffea83fec05ade9f207/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.560781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebbe533568e48b1bdba51e70eb969eb, user_id=037bc1ac03554ffea83fec05ade9f207, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.563457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.634571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 938/1881] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:09 2022] PUT /identity/v3/projects/0ebbe533568e48b1bdba51e70eb969eb/users/037bc1ac03554ffea83fec05ade9f207/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:09.639305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.649394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411ed5-1e13-4104-9c2a-2c6def4dfd8c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.705700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.706369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.706656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.706934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.707663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.709287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39ef5-b169-41a5-b278-e257d0cec9c6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.718906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 944/1882] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:09.723047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.733489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c48f34-4546-4860-b2ef-aaa584602a52 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.738085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.738963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.739245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.739522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] PATH_INFO: `/v3/projects/0ebbe533568e48b1bdba51e70eb969eb/users/037bc1ac03554ffea83fec05ade9f207/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.741552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebbe533568e48b1bdba51e70eb969eb, user_id=037bc1ac03554ffea83fec05ade9f207, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.744185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.750427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:09.755607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 939/1883] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:09 2022] PUT /identity/v3/projects/0ebbe533568e48b1bdba51e70eb969eb/users/037bc1ac03554ffea83fec05ade9f207/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:09.767076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.767398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.767691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.793741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] MFA Rules not processed for user `037bc1ac03554ffea83fec05ade9f207`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:09.794955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:09.836289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:09.866090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 945/1884] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:09.878988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:09.880681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:09.943602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352fee4-f5b5-45f0-ad4f-119911040bc2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:09.947356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:09.947992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:09.948287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:09.949178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:09.963050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:09.963050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:09.970888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 940/1885] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:10.658236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:10.661913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:10.776510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e15a2-c1eb-4dca-9d05-5233b4fb5c58 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:10.782086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:10.782969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:10.783352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:10.783680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] PATH_INFO: `/v3/users/2b4fc16a6d6f4c94b8767dbc423c32a4` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:10.785095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4fc16a6d6f4c94b8767dbc423c32a4)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:10.787147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:10.909876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:10.921305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190169c-d37d-4631-85b4-af2a51d79f5c service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:10.927226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '037bc1ac03554ffea83fec05ade9f207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ebbe533568e48b1bdba51e70eb969eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:10.927895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:10.928186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:10.928469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:10.929305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:10.929643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:10.931273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:10.947769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 941/1886] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:40:10.948860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:10.959346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b1fbb-0917-4b37-8998-1ac7c7630120 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:10.963994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '037bc1ac03554ffea83fec05ade9f207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ebbe533568e48b1bdba51e70eb969eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:10.965004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:10.965367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:10.965627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:10.966410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:10.968697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:10.988870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4fc16a6d6f4c94b8767dbc423c32a4'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:10.989688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4fc16a6d6f4c94b8767dbc423c32a4'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:10.990419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4fc16a6d6f4c94b8767dbc423c32a4'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:10.998939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4fc16a6d6f4c94b8767dbc423c32a4'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.011965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a0eca2be-3b7f-4e76-a891-88ff3c461446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4fc16a6d6f4c94b8767dbc423c32a4'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.059966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 942/1887] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:10 2022] POST /identity/v3/projects => generated 353 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:11.064144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.065869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.111647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 946/1888] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:10 2022] DELETE /identity/v3/users/2b4fc16a6d6f4c94b8767dbc423c32a4 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:11.116362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1957-6f2b-4377-af02-982605e8189e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.116591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1957-6f2b-4377-af02-982605e8189e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.116876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe1957-6f2b-4377-af02-982605e8189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.135491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-babe1957-6f2b-4377-af02-982605e8189e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:11.136713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-babe1957-6f2b-4377-af02-982605e8189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.138583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e02d-a00a-4c66-9dbc-f45a2feadb02 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.198215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:11.216665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 947/1889] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:11.217983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '037bc1ac03554ffea83fec05ade9f207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ebbe533568e48b1bdba51e70eb969eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.218672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.221349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.221349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.221349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:11.221349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.222338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:11.227096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe1957-6f2b-4377-af02-982605e8189e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.228513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-babe1957-6f2b-4377-af02-982605e8189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.245449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 943/1890] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:40:11.289373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.300976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445f44d-5294-46e3-abf9-02974a107a34 tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.308057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '037bc1ac03554ffea83fec05ade9f207', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ebbe533568e48b1bdba51e70eb969eb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.309513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.309878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.310174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] PATH_INFO: `/v3/projects/e5dd747aa67d4335813e227153380edc` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.310808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe1957-6f2b-4377-af02-982605e8189e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.311533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e5dd747aa67d4335813e227153380edc)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.313625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:11.315575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.315737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:11.316161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.316647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.316978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.329742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.331578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:11.345788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 948/1891] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:11.447585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5dd747aa67d4335813e227153380edc', 'request_id': 'req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.454250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5dd747aa67d4335813e227153380edc', 'request_id': 'req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.554509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5dd747aa67d4335813e227153380edc', 'request_id': 'req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.568010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] The token cache is being invalidate because project e5dd747aa67d4335813e227153380edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:11.568311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.568985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:11.570662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 944/1892] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:11 2022] DELETE /identity/v3/projects/e5dd747aa67d4335813e227153380edc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:11.888179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.889809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.903431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b46fc-9952-43d2-b573-98ce82ed4744 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.967002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.968712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.985839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.988437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.988437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.988437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] PATH_INFO: `/v3/projects/6ed6f56f8a284fc990994a2467429233` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.988723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed6f56f8a284fc990994a2467429233)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.990669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:12.057236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b7ae5-3bc9-4c68-8cb6-fc1da50ac3de tempest-VolumeQuotaClassesTest-1111352510 tempest-VolumeQuotaClassesTest-1111352510-project-admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:12.063058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:12.064119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.067334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.067334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] PATH_INFO: `/v3/users/2104e109cc514e5997aed1401bdb3af7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.067334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2104e109cc514e5997aed1401bdb3af7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:12.067969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:12.163345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed6f56f8a284fc990994a2467429233', 'request_id': 'req-eb48da31-1d64-4716-9541-a62ec87bfb01'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.173409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed6f56f8a284fc990994a2467429233', 'request_id': 'req-eb48da31-1d64-4716-9541-a62ec87bfb01'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.246233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed6f56f8a284fc990994a2467429233', 'request_id': 'req-eb48da31-1d64-4716-9541-a62ec87bfb01'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.263894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] The token cache is being invalidate because project 6ed6f56f8a284fc990994a2467429233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:12.264255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.265625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.268385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 949/1893] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:11 2022] DELETE /identity/v3/projects/6ed6f56f8a284fc990994a2467429233 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:12.271461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:12.273004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.276600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2104e109cc514e5997aed1401bdb3af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.277400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2104e109cc514e5997aed1401bdb3af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.278150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2104e109cc514e5997aed1401bdb3af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.286489 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2104e109cc514e5997aed1401bdb3af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.299244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2104e109cc514e5997aed1401bdb3af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.306875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:12.349191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48da31-1d64-4716-9541-a62ec87bfb01 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:12.351722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:12.352616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.352900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.353204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] PATH_INFO: `/v3/users/1e6f2ad894004ec58ef34fed1a5bb689` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.354446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6f2ad894004ec58ef34fed1a5bb689)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:12.357102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:12.375165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 945/1894] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:11 2022] DELETE /identity/v3/users/2104e109cc514e5997aed1401bdb3af7 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:12.376031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:12.377454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.389957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:12.443444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3700-9898-451d-8b49-2884b474db7f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:12.448285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:12.449182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.449492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.449786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] PATH_INFO: `/v3/users/a7036b95a322471d887f8b1146e1bf3e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.451291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7036b95a322471d887f8b1146e1bf3e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:12.453352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:12.528538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6f2ad894004ec58ef34fed1a5bb689'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.529239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6f2ad894004ec58ef34fed1a5bb689'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.529935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6f2ad894004ec58ef34fed1a5bb689'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.537536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6f2ad894004ec58ef34fed1a5bb689'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.561118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2413d2cb-a937-4896-85ca-9211b949c3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6f2ad894004ec58ef34fed1a5bb689'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.610178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7036b95a322471d887f8b1146e1bf3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.611728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7036b95a322471d887f8b1146e1bf3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.612921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7036b95a322471d887f8b1146e1bf3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.620619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7036b95a322471d887f8b1146e1bf3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.645659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6f265214-7e79-4ed7-8523-7167e31bc076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7036b95a322471d887f8b1146e1bf3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:12.645946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 950/1895] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:12 2022] DELETE /identity/v3/users/1e6f2ad894004ec58ef34fed1a5bb689 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:12.645946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.645946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.645946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.681993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:12.683541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.727911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 946/1896] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:12 2022] DELETE /identity/v3/users/a7036b95a322471d887f8b1146e1bf3e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:12.734174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.734587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.735871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.766694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 951/1897] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:12.767459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:12.769051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.777696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:12.779682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.854463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 947/1898] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:12.865096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:12.867945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:12.884415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c052e-5953-4e7b-98b1-450f51c0d072 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:12.945500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:12.946252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.946448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.946731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.954054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7d936-7850-45af-8859-a922be72fa61 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:12.958174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:12.958837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:12.959326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:12.959614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:12.961447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:12.963187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:12.979134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 952/1899] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:12.982629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:12.984139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.000774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 948/1900] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:13.131982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:13.189487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:13.291115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:13.292541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.309861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa63db-2ae8-403e-931b-1ecd49fac98d service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:13.316642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:13.317840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.317966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.318241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] PATH_INFO: `/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.319480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f23a2b72494e34b7a4c3d2e2ce7b05)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:13.321448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.461149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f23a2b72494e34b7a4c3d2e2ce7b05', 'request_id': 'req-3879f742-a980-4f44-b1c4-b45fc68cb511'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.497481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f23a2b72494e34b7a4c3d2e2ce7b05', 'request_id': 'req-3879f742-a980-4f44-b1c4-b45fc68cb511'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.538840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:13.540502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.554053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4786-07e0-44cf-a82d-22436c87dce3 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:13.615722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:13.616541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.616828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.617108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] PATH_INFO: `/v3/projects/1d88d7d001a44867bd926bf90fa128d1` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.617391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f23a2b72494e34b7a4c3d2e2ce7b05', 'request_id': 'req-3879f742-a980-4f44-b1c4-b45fc68cb511'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.618365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d88d7d001a44867bd926bf90fa128d1)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:13.620300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.630229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] The token cache is being invalidate because project 19f23a2b72494e34b7a4c3d2e2ce7b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:13.630528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.631195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3879f742-a980-4f44-b1c4-b45fc68cb511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.632925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 953/1901] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:13 2022] DELETE /identity/v3/projects/19f23a2b72494e34b7a4c3d2e2ce7b05 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:13.638258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bddd1a-766f-4875-9862-692df3646495 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.638612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bddd1a-766f-4875-9862-692df3646495 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.638898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bddd1a-766f-4875-9862-692df3646495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.657229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d4bddd1a-766f-4875-9862-692df3646495 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:13.658429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d4bddd1a-766f-4875-9862-692df3646495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.666367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 954/1902] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:13.674067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.674351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.674625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.693479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:13.694690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.711983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 955/1903] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:13.715677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:13.717117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.734300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3f89b-92bb-4814-b13f-89ac0349e4c1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:13.738873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d88d7d001a44867bd926bf90fa128d1', 'request_id': 'req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.744565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d88d7d001a44867bd926bf90fa128d1', 'request_id': 'req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.791917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:13.792821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.797004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.797004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] PATH_INFO: `/v3/projects/98d908d54689431e8c972da268238f6d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.797004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d908d54689431e8c972da268238f6d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:13.797004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.816542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d88d7d001a44867bd926bf90fa128d1', 'request_id': 'req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.828543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] The token cache is being invalidate because project 1d88d7d001a44867bd926bf90fa128d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:13.828838 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.829592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.831244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 949/1904] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:13 2022] DELETE /identity/v3/projects/1d88d7d001a44867bd926bf90fa128d1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:13.834259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:13.834259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.846248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f1b0a-8cf4-47c9-9ecd-5f94775924cf admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:13.851897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:13.852563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.855113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.855113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.855113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:13.855113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:13.856374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.868259 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 950/1905] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:13.869289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:13.870899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.884296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be586-c9c6-4a03-a6db-db5eb21ab850 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:13.888916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:13.889817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.890103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.890377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] PATH_INFO: `/v3/users/037bc1ac03554ffea83fec05ade9f207` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.891561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=037bc1ac03554ffea83fec05ade9f207)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:13.893466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:13.949362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d908d54689431e8c972da268238f6d', 'request_id': 'req-a09bd219-f46b-4f14-b881-a3dddb3cab73'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:13.954992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d908d54689431e8c972da268238f6d', 'request_id': 'req-a09bd219-f46b-4f14-b881-a3dddb3cab73'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.051941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d908d54689431e8c972da268238f6d', 'request_id': 'req-a09bd219-f46b-4f14-b881-a3dddb3cab73'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.064250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] The token cache is being invalidate because project 98d908d54689431e8c972da268238f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:14.064565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.065252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.066873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 956/1906] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:13 2022] DELETE /identity/v3/projects/98d908d54689431e8c972da268238f6d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:14.067803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.069174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:14.122543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037bc1ac03554ffea83fec05ade9f207'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.123440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037bc1ac03554ffea83fec05ade9f207'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.124063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037bc1ac03554ffea83fec05ade9f207'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.130913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037bc1ac03554ffea83fec05ade9f207'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.132368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd219-f46b-4f14-b881-a3dddb3cab73 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.137075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.137792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.138078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.138356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.139054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.140955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.151640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037bc1ac03554ffea83fec05ade9f207'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.241394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 951/1907] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:13 2022] DELETE /identity/v3/users/037bc1ac03554ffea83fec05ade9f207 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:14.242522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.249512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 957/1908] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:14 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:14.251067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.256266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:14.258007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:14.358144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba44ad8-88dd-47f3-87a1-0625a56516c6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.364574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8254-f4d4-4859-b884-9d0403b467e5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.366551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.368020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.368751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.370144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] PATH_INFO: `/v3/users/8caca919fc8147d29f18c91cef54d447` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.379244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8caca919fc8147d29f18c91cef54d447)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.380635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.381366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.382690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.382690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.382889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.382978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.387372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.493137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 958/1909] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:14 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:14.497733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.521842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68b18-5e62-4940-817a-79147dc18410 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.532521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.533794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.534028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.534449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.535517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.537702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.571288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 959/1910] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:14.575174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.587015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebc831-af16-4602-ac76-9b8939fded84 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.595387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.596477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.596971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.597476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.598499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.600502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.607473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8caca919fc8147d29f18c91cef54d447'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.608287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8caca919fc8147d29f18c91cef54d447'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.609062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8caca919fc8147d29f18c91cef54d447'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.614751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 960/1911] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:14.617652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8caca919fc8147d29f18c91cef54d447'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.618833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.622670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:14.643328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500488d-3d7a-4adf-af39-237fb334d7c7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.649627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47836e59-5b9b-4919-8dce-e747f0ace6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8caca919fc8147d29f18c91cef54d447'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:14.653981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.656188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.657181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.657181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] PATH_INFO: `/v3/projects/02bd39ab4289402788492b87ef1ddbcd/users/c76f29d1984e42758f8d97f65b7acfe1/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.674702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=02bd39ab4289402788492b87ef1ddbcd, user_id=c76f29d1984e42758f8d97f65b7acfe1, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.678174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:14.697962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:14.723502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 952/1912] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:14 2022] DELETE /identity/v3/users/8caca919fc8147d29f18c91cef54d447 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:14.739576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 961/1913] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:14 2022] PUT /identity/v3/projects/02bd39ab4289402788492b87ef1ddbcd/users/c76f29d1984e42758f8d97f65b7acfe1/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:14.754126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.754536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.754825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.789766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] MFA Rules not processed for user `c76f29d1984e42758f8d97f65b7acfe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:14.792200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:14.803256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:14.898883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 953/1914] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:14.903528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:14.979073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1a092-4435-4709-876e-df4ff8763fcd admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:14.985891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:14.986585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:14.986873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:14.987697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:14.988433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:14.991385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.009050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.010431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.034536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a826-ab15-4f5d-ba02-3efeb4a98f7e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.041651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.042825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.043082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.043359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] PATH_INFO: `/v3/projects/0ebbe533568e48b1bdba51e70eb969eb` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.044656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebbe533568e48b1bdba51e70eb969eb)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.046800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.124636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 962/1915] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:14 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:15.127594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.184676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebbe533568e48b1bdba51e70eb969eb', 'request_id': 'req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.191473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebbe533568e48b1bdba51e70eb969eb', 'request_id': 'req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.216429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd6b65-1b16-4138-b936-99ef38949bd2 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.268860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebbe533568e48b1bdba51e70eb969eb', 'request_id': 'req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.277850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.277850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.277850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.277850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.278220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.279491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5fcae-b2c4-435e-9495-301122ab3863 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.285649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] The token cache is being invalidate because project 0ebbe533568e48b1bdba51e70eb969eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:15.286056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.286990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.290586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 954/1916] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:15 2022] DELETE /identity/v3/projects/0ebbe533568e48b1bdba51e70eb969eb => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:15.297393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.299041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.313697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cffdfe-0c36-4cfe-8886-f6b650c1b53a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.318846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.319736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.320016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.320291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] PATH_INFO: `/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.321497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3563a9f411342eea5fbfe6e5b0396a0)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.323872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.368754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 963/1917] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:15.368754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178903e-7283-48a1-9e2d-d0512b84b1c5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.368754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178903e-7283-48a1-9e2d-d0512b84b1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.368754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178903e-7283-48a1-9e2d-d0512b84b1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.388485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c178903e-7283-48a1-9e2d-d0512b84b1c5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:15.389733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c178903e-7283-48a1-9e2d-d0512b84b1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.397842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 964/1918] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:15.399101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52665175-6076-461f-95e9-077bbec975f7 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.399808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52665175-6076-461f-95e9-077bbec975f7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.399808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52665175-6076-461f-95e9-077bbec975f7 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.400913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 965/1919] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:40:15.401645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52665175-6076-461f-95e9-077bbec975f7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.403262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-52665175-6076-461f-95e9-077bbec975f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.419097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52665175-6076-461f-95e9-077bbec975f7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.425714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.426309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.428485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.428485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.428485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.429405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c6a3d-27f5-45a8-8377-e2cb4b4c9ad0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.441421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 966/1920] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:15.442576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8284-7b71-4933-aa24-f2dc0085c720 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.442868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8284-7b71-4933-aa24-f2dc0085c720 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.443243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8284-7b71-4933-aa24-f2dc0085c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.462816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7e5a8284-7b71-4933-aa24-f2dc0085c720 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:15.464028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7e5a8284-7b71-4933-aa24-f2dc0085c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.474050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3563a9f411342eea5fbfe6e5b0396a0', 'request_id': 'req-35044d09-d34e-4daf-8dd3-22c0194a5156'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.481774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3563a9f411342eea5fbfe6e5b0396a0', 'request_id': 'req-35044d09-d34e-4daf-8dd3-22c0194a5156'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.566616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 967/1921] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:15.568048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeda63-f67c-4700-839d-e6e78ae3d30f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.568471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeda63-f67c-4700-839d-e6e78ae3d30f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.568762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeda63-f67c-4700-839d-e6e78ae3d30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.589692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c0eeda63-f67c-4700-839d-e6e78ae3d30f None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:15.590918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c0eeda63-f67c-4700-839d-e6e78ae3d30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.596569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3563a9f411342eea5fbfe6e5b0396a0', 'request_id': 'req-35044d09-d34e-4daf-8dd3-22c0194a5156'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.611747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] The token cache is being invalidate because project b3563a9f411342eea5fbfe6e5b0396a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:15.612152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.613023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:15.616351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 955/1922] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:15 2022] DELETE /identity/v3/projects/b3563a9f411342eea5fbfe6e5b0396a0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:15.619815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.629710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.653139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35044d09-d34e-4daf-8dd3-22c0194a5156 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.674662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.675618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.675994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.676272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.677023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.678848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.704237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 956/1923] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:15.704960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.710832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.724974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 968/1924] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:15.726837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.727340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.727803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.727990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d43d6-8ae7-4b76-953c-60413db50932 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.734933 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.735971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.736187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.736547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.737353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:15.737663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.739334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.750154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:15.751377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.755175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 957/1925] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:15.756319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.765053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 969/1926] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:15.771841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.782490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27830e-eae4-478b-a173-0dd55b7e8ea7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.790347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.792915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.792915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.792915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] PATH_INFO: `/v3/projects/febe0385b87e4b0c9a9dafa5aaae1b08/users/9c643709fdcd42129c937b9e5e950efe/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.796871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af3d8-8580-425e-a76d-964af953cd58 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:15.811747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] RBAC: Authorizing `identity:create_grant(project_id=febe0385b87e4b0c9a9dafa5aaae1b08, user_id=9c643709fdcd42129c937b9e5e950efe, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.814078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231cf71-167c-42e0-a334-4d852d6a929a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.817938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:15.818617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.819511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.819796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.820490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:15.822490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:15.886620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 958/1927] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:15 2022] PUT /identity/v3/projects/febe0385b87e4b0c9a9dafa5aaae1b08/users/9c643709fdcd42129c937b9e5e950efe/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:15.899219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:15.899556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:15.899782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:15.910840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 970/1928] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:15.913284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:15.913477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:15.936652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:15.938377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:16.006289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2670944-1258-4080-91a3-b8f383e2d715 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.013098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.014584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.014700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.015163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.024628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.026340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e487bc-895c-4e3d-97d9-49443f11a9d1 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.038606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 971/1929] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:16.040332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.040638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.040918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.060599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 959/1930] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:16.061460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.074681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] MFA Rules not processed for user `9c643709fdcd42129c937b9e5e950efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:16.076572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:16.082656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af682e5d-e7da-473f-9604-53685c4270ad None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.097674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.099611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.099910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.100199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.102665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.104934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.163449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 972/1931] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:16.163449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.186763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c191c3-ee09-4333-9fe8-caf5bbf70f5c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.192161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.192906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.193399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.193689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.194503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:16.194815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.196433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.212068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 960/1932] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:16.215952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.220443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 973/1933] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:16.222294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.228610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc4efa-508b-4560-a316-8b5f8c3e90d7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.232853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285900-1bcf-4d04-a2f1-e02f4250ae05 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.233860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.234822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.235113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.235538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.236248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.238184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.244011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.244011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.244011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.244011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.244384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.246419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.272785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 974/1934] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:16.272785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.281721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92ffc-4e29-4d73-b5b7-4d56b873523e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.286578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.287306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.287661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.288009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.288783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.290816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.327488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:16.333989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 961/1935] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:16.333989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.407209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26743-7c9e-4367-b3f8-6797ff4810d3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.413117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.413934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.414320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.414851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.415798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.418092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.443430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 962/1936] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:16.444652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.449034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 975/1937] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:16.453324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.459447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e4ed5-f47e-4db7-86bb-eb50e9997c70 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.500587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:16.540114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.541019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.541426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.541793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.542296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b887f6b-bd85-4fbb-b531-bc593b498a10 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.542604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.544538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.547188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.548547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.548547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.548865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] PATH_INFO: `/v3/projects/d09555e876d14361a8a7c6152de17a45/users/cb10e737df2947fba54620faf0b75081/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.565118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09555e876d14361a8a7c6152de17a45, user_id=cb10e737df2947fba54620faf0b75081, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.567807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.623948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 963/1938] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:16.624891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.636637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 976/1939] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:16 2022] PUT /identity/v3/projects/d09555e876d14361a8a7c6152de17a45/users/cb10e737df2947fba54620faf0b75081/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:16.636906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c117331-bb66-4c00-b353-e0db063ff834 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.649866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.700410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.701234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.701604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.701947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.702712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.704748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.714955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc05d28-2bb2-4b2f-94f0-4d1c99922397 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.719765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.720428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.720711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.720989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.721744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.723324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c32adf-e61b-46ff-a275-be404dbb8522 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.736779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 977/1940] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:16.738709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.739146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.739818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.773045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] MFA Rules not processed for user `cb10e737df2947fba54620faf0b75081`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:16.774305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:16.799757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 964/1941] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:16.799757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.810086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba0c82-38d1-40f4-acbf-2a214f859a11 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.816544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.817713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.818569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.818569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] PATH_INFO: `/v3/projects/459f7c5ac99a41678a2363b0972ae2ee/users/56d993d9002a40b49b9d3032bd458708/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.838997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459f7c5ac99a41678a2363b0972ae2ee, user_id=56d993d9002a40b49b9d3032bd458708, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.842511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.867412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 978/1942] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:16.870695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.890507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40206742-07b9-4de9-8e5d-3721efbfe762 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.895099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.896225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.896511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.896791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.897562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.899162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.906830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 965/1943] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:16 2022] PUT /identity/v3/projects/459f7c5ac99a41678a2363b0972ae2ee/users/56d993d9002a40b49b9d3032bd458708/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:16.910242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 979/1944] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:16.911088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.913980 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.916082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:16.925280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eca40-7e61-4fbf-b24b-be0ec989cbf7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.989421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ef-1d3b-47a9-a9ef-9132d3fc3e66 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.993000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.993782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.994060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.994338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.994503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.995637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.995637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.995841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.996545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.999630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.012917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 980/1945] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:17.013657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.027758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97340a2-7532-4be4-8372-a14b6048d531 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.035433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.036432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.036958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.037794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.038553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.041182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.068673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.070818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 981/1946] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:17.071095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.075083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.082785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c16fc-207d-4d0f-b52c-e515be0ec7ab admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.087249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 966/1947] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:17.088124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.088426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.090761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.090761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.090761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] PATH_INFO: `/v3/projects/459f7c5ac99a41678a2363b0972ae2ee/users/56d993d9002a40b49b9d3032bd458708/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.092740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459f7c5ac99a41678a2363b0972ae2ee, user_id=56d993d9002a40b49b9d3032bd458708, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.095994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57d96e-1438-4601-8fde-28afa1559339 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.101287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c8ad2-8cd3-479d-8255-3ffacfc95728 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.109748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.110924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.111221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.111503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] PATH_INFO: `/v3/projects/167da863020c4ed7b617191f60d60511/users/3a3d526207cd48e0994bdd411bd0c889/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.113240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 982/1948] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:17 2022] PUT /identity/v3/projects/459f7c5ac99a41678a2363b0972ae2ee/users/56d993d9002a40b49b9d3032bd458708/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.123030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.123321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.123648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.128216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=167da863020c4ed7b617191f60d60511, user_id=3a3d526207cd48e0994bdd411bd0c889, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.131194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d7134-f50a-4234-9a90-9444f6082494 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.140686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.156528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] MFA Rules not processed for user `56d993d9002a40b49b9d3032bd458708`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:17.157928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.200538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 967/1949] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:17 2022] PUT /identity/v3/projects/167da863020c4ed7b617191f60d60511/users/3a3d526207cd48e0994bdd411bd0c889/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.212882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.213515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.213515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.235556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.246330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 983/1950] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:17.248794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] MFA Rules not processed for user `3a3d526207cd48e0994bdd411bd0c889`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:17.250053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.267501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.269474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.331079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543b903-60a7-4e5d-99e6-986be907638c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.337068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 968/1951] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:17.343766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.394470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.395134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.395423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.395700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.397652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.418627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39289c1-077f-44af-a691-f273121d3c82 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.428346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.428904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.429361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.429711 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.430616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.432738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.473646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.475080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.483940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 984/1952] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:17.543989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 969/1953] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:17 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:17.548076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.559747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5298ab8-d26f-45eb-b5f4-a6776c17a80f service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.612408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.622433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.623395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.623679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.623952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.624654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.626597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.704087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 985/1954] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:17 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:17.707510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.725453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bcf89-d0a0-4c6b-b565-7aba1f6c5642 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.731205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.732185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.732185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.732473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.733247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.734849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.748295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 970/1955] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:17.751156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.761381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87c88-4f77-48f9-8218-4aeb8676e577 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.767435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.768323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.768610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.779808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] PATH_INFO: `/v3/projects/9557b90c03ca46b3b13a7a107fad53da/users/46933afeae4f483db85d89e3db3ba52d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.784511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9557b90c03ca46b3b13a7a107fad53da, user_id=46933afeae4f483db85d89e3db3ba52d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.787373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.885841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 986/1956] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:17 2022] PUT /identity/v3/projects/9557b90c03ca46b3b13a7a107fad53da/users/46933afeae4f483db85d89e3db3ba52d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.890377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.900344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65d27c-9a22-4ae2-8210-5b75a3850089 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.903264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.960555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.961258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.961543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.961815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.962555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.964181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3e293-73dd-4c0c-9d65-918bb3bb580b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.974436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 971/1957] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:17.978590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.989749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f668095-f0fe-4caa-8666-3d0d63a749dd admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.991552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.995900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.996781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.998014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.998014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] PATH_INFO: `/v3/projects/9557b90c03ca46b3b13a7a107fad53da/users/46933afeae4f483db85d89e3db3ba52d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.999615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9557b90c03ca46b3b13a7a107fad53da, user_id=46933afeae4f483db85d89e3db3ba52d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:18.002288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:18.012285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 987/1958] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:17 2022] PUT /identity/v3/projects/9557b90c03ca46b3b13a7a107fad53da/users/46933afeae4f483db85d89e3db3ba52d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:18.022849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:18.023180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:18.023775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:18.052818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] MFA Rules not processed for user `46933afeae4f483db85d89e3db3ba52d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:18.054136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:18.154005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 972/1959] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:18.178126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:18.229094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:18.238121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb38e-1cf4-4241-ac9b-0185e4064e21 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:18.242249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:18.242840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:18.243169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:18.243705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:18.256359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:18.257937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e7ce-ef50-471f-bc3f-931b55a60c05 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:18.281594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 988/1960] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:18.371835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:18.553439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:18.577883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:18.831034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:18.892856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea8dfa-8439-4b1e-9b31-1814686bc57f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:18.897428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:18.898234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:18.898516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:18.898796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] PATH_INFO: `/v3/users/3a3d526207cd48e0994bdd411bd0c889` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:18.899975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3d526207cd48e0994bdd411bd0c889)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:18.901944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:19.080588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3d526207cd48e0994bdd411bd0c889'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:19.081287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3d526207cd48e0994bdd411bd0c889'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:19.081905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3d526207cd48e0994bdd411bd0c889'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:19.089027 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3d526207cd48e0994bdd411bd0c889'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:19.090226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:19.103084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3d526207cd48e0994bdd411bd0c889'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:19.170817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 973/1961] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:18 2022] DELETE /identity/v3/users/3a3d526207cd48e0994bdd411bd0c889 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:19.175293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:19.175585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:19.175873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:19.194105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:19.195330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:19.221546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:19.254621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 989/1962] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:19.264640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:19.266052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:19.335116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913f040-9cf3-4041-b2a0-f294bb8722b8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:19.338765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:19.339419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:19.339708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:19.339987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:19.349992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:19.351426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:19.361340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 974/1963] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:19.858509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:19.859947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:19.881312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cb216-5d72-46da-b66a-e737f5c366bd None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:19.887295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:19.888046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:19.888389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:19.888712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] PATH_INFO: `/v3/projects/167da863020c4ed7b617191f60d60511` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:19.889892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=167da863020c4ed7b617191f60d60511)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:19.891786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:20.022305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167da863020c4ed7b617191f60d60511', 'request_id': 'req-952033b3-1b5b-42fd-8993-444c4588fbea'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.027656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167da863020c4ed7b617191f60d60511', 'request_id': 'req-952033b3-1b5b-42fd-8993-444c4588fbea'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.142939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167da863020c4ed7b617191f60d60511', 'request_id': 'req-952033b3-1b5b-42fd-8993-444c4588fbea'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.155400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] The token cache is being invalidate because project 167da863020c4ed7b617191f60d60511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:20.155692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.156358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.157946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 990/1964] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:19 2022] DELETE /identity/v3/projects/167da863020c4ed7b617191f60d60511 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:20.161818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:20.172050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa339d-7026-4228-96c6-91643ade40ef service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:20.226929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:20.227803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:20.228090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:20.228370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] PATH_INFO: `/v3/users/46933afeae4f483db85d89e3db3ba52d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:20.229626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=46933afeae4f483db85d89e3db3ba52d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:20.231548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:20.460467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46933afeae4f483db85d89e3db3ba52d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.461454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46933afeae4f483db85d89e3db3ba52d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.462383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46933afeae4f483db85d89e3db3ba52d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.473836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46933afeae4f483db85d89e3db3ba52d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.489917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46933afeae4f483db85d89e3db3ba52d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:20.493737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:20.562683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 975/1965] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:20 2022] DELETE /identity/v3/users/46933afeae4f483db85d89e3db3ba52d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:20.577323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:20.578850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:20.663765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952033b3-1b5b-42fd-8993-444c4588fbea admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:20.727005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:20.727539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:20.727828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:20.728109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:20.729963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:20.768605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:20.829858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:20.833811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabd92f-5460-4ccd-9b14-8bb3e603c4fa service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:20.848670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 991/1966] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:21.171657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:21.172145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:21.256744 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c2c7a-78af-4f89-afc7-16033a0c02ad admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:21.261494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:21.262354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.262643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.262912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] PATH_INFO: `/v3/projects/9557b90c03ca46b3b13a7a107fad53da` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.264703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9557b90c03ca46b3b13a7a107fad53da)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:21.266680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:21.377132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9557b90c03ca46b3b13a7a107fad53da', 'request_id': 'req-176e5aa8-35f1-4248-8061-f6a26dc1d287'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:21.382564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9557b90c03ca46b3b13a7a107fad53da', 'request_id': 'req-176e5aa8-35f1-4248-8061-f6a26dc1d287'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:21.450407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9557b90c03ca46b3b13a7a107fad53da', 'request_id': 'req-176e5aa8-35f1-4248-8061-f6a26dc1d287'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:21.464687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] The token cache is being invalidate because project 9557b90c03ca46b3b13a7a107fad53da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:21.465304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:21.465773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-176e5aa8-35f1-4248-8061-f6a26dc1d287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:21.468217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 976/1967] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:21 2022] DELETE /identity/v3/projects/9557b90c03ca46b3b13a7a107fad53da => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:21.473260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.473703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.476564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.494253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:21.495617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:21.564922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 992/1968] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:21.573793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.574173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.574528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.597094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:21.598679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:21.606764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 977/1969] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:21.611267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:21.621149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0718-9810-45e3-8a87-c7b9f50855d7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:21.626391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:21.627056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.627344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.627623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.628465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:21.628781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:21.630431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:21.643100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 993/1970] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:21.650320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:21.661281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332eae-f9ab-47b2-b6b2-af93b7597679 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:21.666202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:21.666868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.667347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.667692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.668392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:21.670367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:21.766936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 978/1971] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:21 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:21.771536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:21.782812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd770eb-3088-423f-a484-c054a02bcf60 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:21.855129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:21.856130 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.856421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.856947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.858953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:21.861054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:21.961412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 994/1972] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:21 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:21.970842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:21.990020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8695-003e-4c14-a885-1eaee5cc8119 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:21.994945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:21.995640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.996141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.996282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.996994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:21.999380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.010840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 979/1973] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:22.015571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.026032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3405eb-0342-4f99-b10d-e13d5fc99d2d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.030719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.031393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.031682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.031959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.032703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.034344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74d6d-5c7e-4df5-b5fc-5ff73209d63f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.045619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 995/1974] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:22.049681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.061967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6e79f-538a-4766-bf60-11465bb511e3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.066708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.067622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.067909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.068189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] PATH_INFO: `/v3/projects/a84eed2a5a3b43469e213a9215356de5/users/1bdb78c00cb142df8affc8a5d56dcca7/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.085728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84eed2a5a3b43469e213a9215356de5, user_id=1bdb78c00cb142df8affc8a5d56dcca7, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.088516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.172149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 980/1975] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:22 2022] PUT /identity/v3/projects/a84eed2a5a3b43469e213a9215356de5/users/1bdb78c00cb142df8affc8a5d56dcca7/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:22.182592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.183052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.183230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.233389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] MFA Rules not processed for user `1bdb78c00cb142df8affc8a5d56dcca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:22.235406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.255606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:22.309774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 996/1976] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:22.322944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.327672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.356473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:22.398273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de55f1-6d5a-48bd-ae9a-e5cb947cfa7a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.406964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.407895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.408383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.408763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.423722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.426252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.436119 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.441363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.451161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 981/1977] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:22.451409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be48cfa-740d-4bc3-90fc-89b08dca15c9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.455357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.456005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.456390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.456680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.458479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.473644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.528428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.531470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0a956-92ea-46fe-8606-ec858e404c1a service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.541497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a5ad8-e39b-4f37-88b8-6f400d2e119c service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.543141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 997/1978] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:22.549021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.550104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.550452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.550788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] PATH_INFO: `/v3/users/1bdb78c00cb142df8affc8a5d56dcca7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.552121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdb78c00cb142df8affc8a5d56dcca7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.554184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.722197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdb78c00cb142df8affc8a5d56dcca7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.722859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdb78c00cb142df8affc8a5d56dcca7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.723481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdb78c00cb142df8affc8a5d56dcca7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.732014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdb78c00cb142df8affc8a5d56dcca7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.744614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdb78c00cb142df8affc8a5d56dcca7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.787832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 982/1979] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:22 2022] DELETE /identity/v3/users/1bdb78c00cb142df8affc8a5d56dcca7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:22.792315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.792806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.792927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.809435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:22.810638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.869952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 998/1980] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:22.880236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.881891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.929569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:22.940154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d31003-85e2-4ca2-bd90-ae832330cba2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.943776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.944420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.944729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.945006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.954774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.956280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab873893-0807-4afc-9395-bbd9aa520886 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.967639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 983/1981] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:23.378606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:23.380248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:23.411733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b086a-b998-4f44-b580-b3cfa93beed0 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:23.417846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:23.418781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.419120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.420023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] PATH_INFO: `/v3/projects/a84eed2a5a3b43469e213a9215356de5` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.432154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84eed2a5a3b43469e213a9215356de5)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:23.434420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:23.551002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84eed2a5a3b43469e213a9215356de5', 'request_id': 'req-87a455cb-665b-4b8f-94f4-35561eeb6fce'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.552526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:23.557982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84eed2a5a3b43469e213a9215356de5', 'request_id': 'req-87a455cb-665b-4b8f-94f4-35561eeb6fce'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.644837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84eed2a5a3b43469e213a9215356de5', 'request_id': 'req-87a455cb-665b-4b8f-94f4-35561eeb6fce'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.665236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] The token cache is being invalidate because project a84eed2a5a3b43469e213a9215356de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:23.665542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.666226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-87a455cb-665b-4b8f-94f4-35561eeb6fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.668177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 999/1982] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:23 2022] DELETE /identity/v3/projects/a84eed2a5a3b43469e213a9215356de5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:23.673792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.674607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.675308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.694607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:23.695833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:23.753111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 984/1983] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:23.760616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.760909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.761274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.779186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:23.780371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:23.788201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1000/1984] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:23.792424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:23.802833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4fed5-7044-4eea-8b2c-5080c6266ff0 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:23.807661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:23.808378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.808704 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.808973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.809781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:23.810621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:23.812291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:23.824442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 985/1985] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:23.828484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:23.837841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fb93d-2e12-47c2-b7a1-814ab44ad09e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:23.843204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:23.843870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.844154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.844428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.845115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:23.847041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:23.960300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1001/1986] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:23.964522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:23.975159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb1a56-4c87-47c7-99a8-dbd0e58b8838 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.032739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.033447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.033735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.034015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.034701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.036622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.112921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 986/1987] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:24.116979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.127923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d504-9da9-48d0-92da-cbd250470d2e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.132518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.133204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.133494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.133774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.134504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.136136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.147018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1002/1988] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:24.150720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.160655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdecdfa-ff14-4ed1-b807-48b24c646835 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.165274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.165928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.166220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.166499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.167249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.168830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5974-9f90-49c6-bb26-ef45571d3769 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.179129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 987/1989] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:24.182912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.193346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4cce2-73ec-4454-a12a-856a7004b212 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.197888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.198748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.199029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.199324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] PATH_INFO: `/v3/projects/999256a922ad4517ae9562d982e86fb6/users/921a487f77754970bbd8903a54335190/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.215060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999256a922ad4517ae9562d982e86fb6, user_id=921a487f77754970bbd8903a54335190, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.218335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.290164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1003/1990] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:24 2022] PUT /identity/v3/projects/999256a922ad4517ae9562d982e86fb6/users/921a487f77754970bbd8903a54335190/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:24.301106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.301440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.301719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.330346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] MFA Rules not processed for user `921a487f77754970bbd8903a54335190`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:24.331571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:24.371330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:24.394187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 988/1991] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:24.398712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.472689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b19bb-1c11-4b6e-817f-1f31d7ad5562 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.477896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.478558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.478844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.479115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.479817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.481813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.591391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1004/1992] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:24.594992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.609695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f70d2-5e27-453c-aa68-ba16b72bf0bf None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.680073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.680752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.681039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.681356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.682061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.684008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.762030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 989/1993] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:24.766975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.777570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7c8b2-e79c-49bf-8afd-ede41c127ddb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.782280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.782983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.783266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.783596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.784333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.786060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.796916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1005/1994] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:24.805742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.816686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65b28-50aa-45c9-96aa-0de1423436e6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.821449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.822327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.822615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.822894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] PATH_INFO: `/v3/projects/98449885b56f4a71a08eb94dcd004475/users/f2421c260da142748e6e212409fcfd89/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.839932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98449885b56f4a71a08eb94dcd004475, user_id=f2421c260da142748e6e212409fcfd89, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.843045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.943122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 990/1995] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:24 2022] PUT /identity/v3/projects/98449885b56f4a71a08eb94dcd004475/users/f2421c260da142748e6e212409fcfd89/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:24.947142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.957230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b229be-e948-435a-a88d-429bd22f55ae admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:25.002423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:25.014374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:25.015052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:25.015401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:25.015681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:25.016416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:25.018068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b6b5d-e16f-46df-b394-e9bf4bbcb39c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:25.028146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1006/1996] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:25.032174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:25.042620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c624c-35ec-4188-9879-5f23c6dbe4b5 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:25.047804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:25.048716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:25.048996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:25.049461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] PATH_INFO: `/v3/projects/98449885b56f4a71a08eb94dcd004475/users/f2421c260da142748e6e212409fcfd89/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:25.051637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98449885b56f4a71a08eb94dcd004475, user_id=f2421c260da142748e6e212409fcfd89, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:25.054531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:25.065955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 991/1997] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:25 2022] PUT /identity/v3/projects/98449885b56f4a71a08eb94dcd004475/users/f2421c260da142748e6e212409fcfd89/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:25.077105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:25.077433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:25.077922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:25.105742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] MFA Rules not processed for user `f2421c260da142748e6e212409fcfd89`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:25.107146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:25.177028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1007/1998] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:25.187950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:25.189739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:25.247248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a724b-0e64-46d2-a56f-e8069851a684 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:25.251199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:25.251845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:25.252127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:25.252406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:25.301535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:25.311929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:25.313537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:25.322671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 992/1999] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:35.274098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=41 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:35.496045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:36.932739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:36.934980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:37.007313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fff2f5-2c0d-4c5e-8011-b5681ed0b943 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:37.012149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:37.013219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.013471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.013598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] PATH_INFO: `/v3/users/b4b57128eb75430b950c095737ac1822` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.014881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b57128eb75430b950c095737ac1822)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:37.016808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:37.055435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:37.068074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05246b-0a70-4de5-936f-a7d829edd3e7 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:37.075044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:37.076142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.076436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.076717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] PATH_INFO: `/v3/users/921a487f77754970bbd8903a54335190` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.078083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=921a487f77754970bbd8903a54335190)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:37.080478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:37.265583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b57128eb75430b950c095737ac1822'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.267025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b57128eb75430b950c095737ac1822'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.268071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b57128eb75430b950c095737ac1822'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.276620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b57128eb75430b950c095737ac1822'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.295214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ee80ba35-f614-4924-b429-9d80eb5406d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b57128eb75430b950c095737ac1822'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.312399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921a487f77754970bbd8903a54335190'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.313362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921a487f77754970bbd8903a54335190'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.313974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921a487f77754970bbd8903a54335190'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.323888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921a487f77754970bbd8903a54335190'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.337022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-910153e7-4313-4f8a-a32e-31a29fa89fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921a487f77754970bbd8903a54335190'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:37.367323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1008/2000] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:36 2022] DELETE /identity/v3/users/b4b57128eb75430b950c095737ac1822 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:37.372402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.373039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.373604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.394385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:37.396021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:37.446456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 993/2001] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:37 2022] DELETE /identity/v3/users/921a487f77754970bbd8903a54335190 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:37.455966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.457568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.457860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.475453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1009/2002] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:37.486831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:37.491767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:37.493359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:37.494564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:37.513751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:37.565301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 994/2003] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:37.566826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551b0e-ab14-48cb-8e8e-52c446e3f07e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:37.573048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:37.634346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:37.640137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.640137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.640137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.642134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59637a4c-4af5-4f58-a525-aed21c36f3f3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:37.650122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:37.651039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:37.651879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:37.651879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:37.655198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:37.656994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:37.663559 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:37.665007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:37.685204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1010/2004] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:37.692076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 995/2005] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:37.996030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:37.997860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.018273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49edc5-6926-4944-aabd-618fd9aed2f6 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.020496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.021489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.021846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.022194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] PATH_INFO: `/v3/users/c76f29d1984e42758f8d97f65b7acfe1` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.023465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76f29d1984e42758f8d97f65b7acfe1)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.025638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.281775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76f29d1984e42758f8d97f65b7acfe1'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.282743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76f29d1984e42758f8d97f65b7acfe1'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.283358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76f29d1984e42758f8d97f65b7acfe1'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.294205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76f29d1984e42758f8d97f65b7acfe1'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.309730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76f29d1984e42758f8d97f65b7acfe1'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.340658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:38.342251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.377008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064196c-1457-4b5f-95e7-6bdaccba28c8 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.382463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.383462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.383823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.384176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] PATH_INFO: `/v3/projects/f3eed61b06634aeb92e4008722f02898` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.385695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3eed61b06634aeb92e4008722f02898)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.388307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.404539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1011/2006] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:37 2022] DELETE /identity/v3/users/c76f29d1984e42758f8d97f65b7acfe1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:38.407363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:38.410176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.418198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:38.492720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:38.506216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b002-db05-4e66-be51-dd14d9720760 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.510974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.511830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.512145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.512415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] PATH_INFO: `/v3/projects/999256a922ad4517ae9562d982e86fb6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.513665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=999256a922ad4517ae9562d982e86fb6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.515578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.541709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3eed61b06634aeb92e4008722f02898', 'request_id': 'req-e128a727-09db-4ec7-8f17-b141f099df0e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.549723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3eed61b06634aeb92e4008722f02898', 'request_id': 'req-e128a727-09db-4ec7-8f17-b141f099df0e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.610516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999256a922ad4517ae9562d982e86fb6', 'request_id': 'req-e1954e67-fa48-400d-9135-63748093fbaf'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.616609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999256a922ad4517ae9562d982e86fb6', 'request_id': 'req-e1954e67-fa48-400d-9135-63748093fbaf'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.623136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3eed61b06634aeb92e4008722f02898', 'request_id': 'req-e128a727-09db-4ec7-8f17-b141f099df0e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.634993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] The token cache is being invalidate because project f3eed61b06634aeb92e4008722f02898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:38.635284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.635961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e128a727-09db-4ec7-8f17-b141f099df0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.640982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 996/2007] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:38 2022] DELETE /identity/v3/projects/f3eed61b06634aeb92e4008722f02898 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:38.640982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.640982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.640982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.657629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:38.658849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.712879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999256a922ad4517ae9562d982e86fb6', 'request_id': 'req-e1954e67-fa48-400d-9135-63748093fbaf'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.721363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 997/2008] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:38.722128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:38.723809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.731379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] The token cache is being invalidate because project 999256a922ad4517ae9562d982e86fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:38.731912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.733128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.735293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1012/2009] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:38 2022] DELETE /identity/v3/projects/999256a922ad4517ae9562d982e86fb6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:38.736310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:38.739752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.742926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537b542-93e3-4c77-a4ad-e9bf77fdcf5e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.749552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.750433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.750718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.750995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] PATH_INFO: `/v3/users/eaf6da2e26de403788e4821bf051df8a` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.752229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf6da2e26de403788e4821bf051df8a)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.754221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.819649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1954e67-fa48-400d-9135-63748093fbaf admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.823326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.823975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.824272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.824570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.826463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.841207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.842761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.852981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1013/2010] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:38.856879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:38.856879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:38.868332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed925-9512-4f2d-b94f-92b5f20bb1a2 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:38.874475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:38.875491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:38.875780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:38.876061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] PATH_INFO: `/v3/users/f2421c260da142748e6e212409fcfd89` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:38.877534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2421c260da142748e6e212409fcfd89)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:38.879620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:38.979345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf6da2e26de403788e4821bf051df8a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.981642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf6da2e26de403788e4821bf051df8a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.983065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf6da2e26de403788e4821bf051df8a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:38.999669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf6da2e26de403788e4821bf051df8a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.013847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf6da2e26de403788e4821bf051df8a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.070534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2421c260da142748e6e212409fcfd89'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.071217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2421c260da142748e6e212409fcfd89'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.072690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2421c260da142748e6e212409fcfd89'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.085309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 998/2011] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:38 2022] DELETE /identity/v3/users/eaf6da2e26de403788e4821bf051df8a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:39.089635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2421c260da142748e6e212409fcfd89'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.108340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2421c260da142748e6e212409fcfd89'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.114686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:39.176822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1014/2012] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:38 2022] DELETE /identity/v3/users/f2421c260da142748e6e212409fcfd89 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:39.596156 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:39.597779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:39.620889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:39.698997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:39.701147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:39.744760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee701e-b67e-406d-9c01-d55d55bb8c77 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:39.789336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbf73-5cda-479f-acdc-e8399493c0ed admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:39.800569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:39.801913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:39.802187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:39.802569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] PATH_INFO: `/v3/projects/21e91c44cec34af6883bd4f99ff11aad` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:39.804305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e91c44cec34af6883bd4f99ff11aad)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:39.806553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:39.829750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:39.830794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:39.831189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:39.831600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] PATH_INFO: `/v3/projects/02bd39ab4289402788492b87ef1ddbcd` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:39.833028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02bd39ab4289402788492b87ef1ddbcd)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:39.835182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:39.919047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e91c44cec34af6883bd4f99ff11aad', 'request_id': 'req-525bf76c-99e3-41f3-943a-b8229d796f18'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.924667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e91c44cec34af6883bd4f99ff11aad', 'request_id': 'req-525bf76c-99e3-41f3-943a-b8229d796f18'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.947849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02bd39ab4289402788492b87ef1ddbcd', 'request_id': 'req-de9cfc31-5ace-44c7-9786-cfa96e156391'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:39.953200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02bd39ab4289402788492b87ef1ddbcd', 'request_id': 'req-de9cfc31-5ace-44c7-9786-cfa96e156391'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.015658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e91c44cec34af6883bd4f99ff11aad', 'request_id': 'req-525bf76c-99e3-41f3-943a-b8229d796f18'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.029958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] The token cache is being invalidate because project 21e91c44cec34af6883bd4f99ff11aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:40.030246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.031228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.032912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1015/2013] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:39 2022] DELETE /identity/v3/projects/21e91c44cec34af6883bd4f99ff11aad => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:40.033572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.034955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.089812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02bd39ab4289402788492b87ef1ddbcd', 'request_id': 'req-de9cfc31-5ace-44c7-9786-cfa96e156391'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.102876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] The token cache is being invalidate because project 02bd39ab4289402788492b87ef1ddbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:40.103169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.103837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-de9cfc31-5ace-44c7-9786-cfa96e156391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.109799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 999/2014] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:39 2022] DELETE /identity/v3/projects/02bd39ab4289402788492b87ef1ddbcd => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:40.111388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.111699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.111995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.156179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf76c-99e3-41f3-943a-b8229d796f18 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.190411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:40.191712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.214568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:40.298793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.299713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.300103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.300462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] PATH_INFO: `/v3/projects/98449885b56f4a71a08eb94dcd004475` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.306342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98449885b56f4a71a08eb94dcd004475)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.310036 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.348214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1000/2015] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:40.349435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.351136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.365305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0bad5-320a-4ccc-bae2-5dc730d7ebba None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.370431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.371497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.371831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.372398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] PATH_INFO: `/v3/users/9c643709fdcd42129c937b9e5e950efe` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.373701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c643709fdcd42129c937b9e5e950efe)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.375837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.462582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98449885b56f4a71a08eb94dcd004475', 'request_id': 'req-9cbf6559-8e29-48f5-906c-528723d46d81'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.472646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98449885b56f4a71a08eb94dcd004475', 'request_id': 'req-9cbf6559-8e29-48f5-906c-528723d46d81'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.574929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98449885b56f4a71a08eb94dcd004475', 'request_id': 'req-9cbf6559-8e29-48f5-906c-528723d46d81'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.587982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] The token cache is being invalidate because project 98449885b56f4a71a08eb94dcd004475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:40.588515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.589265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9cbf6559-8e29-48f5-906c-528723d46d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.596407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1016/2016] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:40 2022] DELETE /identity/v3/projects/98449885b56f4a71a08eb94dcd004475 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:40.601526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dca46-3dae-4c7f-a2be-2491a3842c16 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.601920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dca46-3dae-4c7f-a2be-2491a3842c16 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.602335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dca46-3dae-4c7f-a2be-2491a3842c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.610038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c643709fdcd42129c937b9e5e950efe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.610741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c643709fdcd42129c937b9e5e950efe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.614232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c643709fdcd42129c937b9e5e950efe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.624312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-129dca46-3dae-4c7f-a2be-2491a3842c16 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:40.625669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-129dca46-3dae-4c7f-a2be-2491a3842c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.636414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c643709fdcd42129c937b9e5e950efe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.654140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2265293e-bcd5-4dce-9271-45864207fe82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c643709fdcd42129c937b9e5e950efe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:40.689259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1017/2017] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:40.692285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.692285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.692285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.709509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:40.710719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.719856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1018/2018] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:40.720661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.730178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1001/2019] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:40 2022] DELETE /identity/v3/users/9c643709fdcd42129c937b9e5e950efe => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:40.730178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.731697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f0ba-1f5c-424b-a452-173311f56a75 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.738715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.740167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.770043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:40.771455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.795439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:40.811302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.811992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.812779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.813226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.814128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:40.814608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.816984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.841713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1019/2020] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:40.844009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.856958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a26af-0bd9-4380-a225-81543540e56d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.862326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1002/2021] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:40.862466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.864560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.864560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.864560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.865466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.867557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.868532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.878394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5bb62-e7e7-4ac4-9475-f950624d3880 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.883304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.884065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.884406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.884805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.885594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:40.885970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.888224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.904529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1003/2022] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:40.905936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:40.915672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047793a2-857a-4948-9bf6-35b4f0679a60 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.920458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.921102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.921424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.921705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.923339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.926294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.001375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1020/2023] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:41.010109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.038905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1004/2024] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:40 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:41.041841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.114553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae038b-2983-4a82-a149-50847c5635b8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.145449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:41.191455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd13c-83a6-48c2-aa8a-7dc5b4fdf794 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.203415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.209325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.209325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.209325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.209325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.209325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.229108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.231495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.232079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.232551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.233466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.235759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.366784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1005/2025] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:41 2022] POST /identity/v3/users => generated 292 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:41.368090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.379776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1021/2026] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:41 2022] POST /identity/v3/users => generated 279 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:41.380934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.383601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989c3f-4bf0-4fec-be3c-4d08e0197554 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.391662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.395543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.395543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.395543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.395543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.396022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.405698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337271-a8b1-42dd-aaeb-cee22acd070d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.411366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.412055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.412345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.412630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.413424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.415264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.425136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1006/2027] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:41.429944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.440892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1022/2028] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:41.442263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737291-52e7-4a24-9184-cdb27d31d56f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.448757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.450335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.451147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.451538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.451927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.453006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.455359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.472077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23b209-6a3d-46fc-a31a-8b1e850eedc6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.488450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.488964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.489336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.489622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.490374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.492281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.505352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1007/2029] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:41.505352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.512706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:41.518474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1023/2030] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:41.519502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.532342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:41.536362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c6e56-764b-4891-9ec3-a98c5a502220 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.545545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.546631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.547078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.547696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] PATH_INFO: `/v3/projects/ee630025d32e4b36b07704a51b7ea0ea/users/3c44d6f249d94133a2b48da40efb08a9/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.550695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e59bf-de47-41c4-86b9-2d64c6c4612f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.567109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:41.568541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.569034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.569549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] PATH_INFO: `/v3/projects/febe0385b87e4b0c9a9dafa5aaae1b08` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.572858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=febe0385b87e4b0c9a9dafa5aaae1b08)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.581008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.620572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee630025d32e4b36b07704a51b7ea0ea, user_id=3c44d6f249d94133a2b48da40efb08a9, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:41.626210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:41.763561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1024/2031] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:41 2022] PUT /identity/v3/projects/ee630025d32e4b36b07704a51b7ea0ea/users/3c44d6f249d94133a2b48da40efb08a9/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:41.764598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:41.787929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febe0385b87e4b0c9a9dafa5aaae1b08', 'request_id': 'req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:41.794845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febe0385b87e4b0c9a9dafa5aaae1b08', 'request_id': 'req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:41.827904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:41.856577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febe0385b87e4b0c9a9dafa5aaae1b08', 'request_id': 'req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:41.859551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782334f9-2ab5-4c0d-a78f-02c1a567735b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:41.871591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] The token cache is being invalidate because project febe0385b87e4b0c9a9dafa5aaae1b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:41.872276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:41.872807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fe3d9a20-8ff5-412e-adb1-0c6786943c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:41.877404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1008/2032] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:41 2022] DELETE /identity/v3/projects/febe0385b87e4b0c9a9dafa5aaae1b08 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:41.877404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:41.877404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:41.877404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:41.927084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] MFA Rules not processed for user `3c44d6f249d94133a2b48da40efb08a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:41.930192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.048591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1009/2033] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:42.048591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.050406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.062415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:42.072270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.074680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.075124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.076851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] PATH_INFO: `/v3/projects/220a5d3b89254edfa065fc9622521866/users/93057a5b9b4d4b87bb960e1a776ac61c/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.117584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220a5d3b89254edfa065fc9622521866, user_id=93057a5b9b4d4b87bb960e1a776ac61c, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.122317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.145575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e792f65-04c7-4e8b-b424-5d6de4701d18 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.151122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.152014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.154311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.154311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] PATH_INFO: `/v3/users/56d993d9002a40b49b9d3032bd458708` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.154473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d993d9002a40b49b9d3032bd458708)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.156362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.211820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1025/2034] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:41 2022] PUT /identity/v3/projects/220a5d3b89254edfa065fc9622521866/users/93057a5b9b4d4b87bb960e1a776ac61c/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:42.213315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.216432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.336026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f911a18-e08a-499a-a38d-106a4ccea2e0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.342115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.342801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.343088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.343370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.344261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.346253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa166d7-f869-412c-9d6c-542256ef9000 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.390948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d993d9002a40b49b9d3032bd458708'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.391857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d993d9002a40b49b9d3032bd458708'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.392578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d993d9002a40b49b9d3032bd458708'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.404727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d993d9002a40b49b9d3032bd458708'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.423522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d993d9002a40b49b9d3032bd458708'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.496785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1026/2035] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:42 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:42.498951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.499321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.499746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.541560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] MFA Rules not processed for user `93057a5b9b4d4b87bb960e1a776ac61c`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:42.542817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.557386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1010/2036] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:42 2022] DELETE /identity/v3/users/56d993d9002a40b49b9d3032bd458708 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:42.558408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.560346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.584994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:42.639737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1027/2037] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:42.643982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.646032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.656143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fc5b-d11e-4ccc-9f77-bf7a6aed9921 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.664367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.666362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.670161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.670161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.670161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.673118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.737621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd2f99-a6cc-4a23-b5a2-229716308201 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.742539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.743206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.743487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.743760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.744461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.747327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.796912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1011/2038] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:42 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:42.802274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.820557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c501-9208-4970-922b-343560421fe4 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.826118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.826847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.830853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.830853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.830853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.830853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.840591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1012/2039] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:42.844978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.851493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:42.855249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c7-142e-45fd-900d-87a7c185f79e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.860052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.860824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.861243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.861655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.862455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.864045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.877391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1013/2040] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:42.882404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.891943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1028/2041] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:42 2022] POST /identity/v3/projects => generated 357 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:42.892193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751db680-574a-4152-a3f9-0a977fb3eba2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.895592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.978814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.978814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.978814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.979112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] PATH_INFO: `/v3/projects/ef03beaddb594300b584f2382f35ac65/users/bbb18368bcde4526a58ed68dccbe37b2/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.993112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0a303-801b-4fd4-8f80-f30d295967a8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.020128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef03beaddb594300b584f2382f35ac65, user_id=bbb18368bcde4526a58ed68dccbe37b2, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.022070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.022942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.023059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.024112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.024685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.025770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.029686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74092841-57ec-4e7e-8bd1-7dd9430ab678 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.103417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1014/2042] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:42 2022] PUT /identity/v3/projects/ef03beaddb594300b584f2382f35ac65/users/bbb18368bcde4526a58ed68dccbe37b2/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:43.103417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.106182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.130939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1029/2043] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:42 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:43.132315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.132608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.132887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.166892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] MFA Rules not processed for user `bbb18368bcde4526a58ed68dccbe37b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:43.168994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.197447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c189-1ddf-45d7-972a-25b3e0bac55f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.204179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.205423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.206807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.207093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] PATH_INFO: `/v3/users/cb10e737df2947fba54620faf0b75081` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.208934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb10e737df2947fba54620faf0b75081)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.211221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.257037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1030/2044] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:43.257791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.268788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:43.270065 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca57b0-e703-4a38-a1aa-381a8b850a41 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.275741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.275903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.276289 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.276482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.277785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.279393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.290938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1031/2045] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:43.291783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.304493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88542cb-2ada-47fc-969e-9e7c70c102f0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.314853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.315671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.316084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.316461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.317566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.319573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.436602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb10e737df2947fba54620faf0b75081'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.443929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb10e737df2947fba54620faf0b75081'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.444915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb10e737df2947fba54620faf0b75081'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.450806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1032/2046] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:43.450806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.457366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.458893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb10e737df2947fba54620faf0b75081'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.472174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb10e737df2947fba54620faf0b75081'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.521676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8e6a-414a-4440-bdb9-6c8633f38cd9 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.526667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.527162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.528046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.528326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] PATH_INFO: `/v3/projects/b831b92653e7451c9c5dfefdcd9cf048/users/9d953ce613954c5aaf49fa1dd8a78d82/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.543753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b831b92653e7451c9c5dfefdcd9cf048, user_id=9d953ce613954c5aaf49fa1dd8a78d82, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.546596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.570076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1015/2047] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:43 2022] DELETE /identity/v3/users/cb10e737df2947fba54620faf0b75081 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:43.571058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.572438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.649035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276bc91-2458-4137-870a-07ed8f174dc7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.654638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1033/2048] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:43 2022] PUT /identity/v3/projects/b831b92653e7451c9c5dfefdcd9cf048/users/9d953ce613954c5aaf49fa1dd8a78d82/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:43.654638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.656052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.715758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.716570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.716851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.717126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] PATH_INFO: `/v3/projects/459f7c5ac99a41678a2363b0972ae2ee` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.718922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459f7c5ac99a41678a2363b0972ae2ee)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.721758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.732115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5082b8-0d4b-4d40-9aee-1bfb18de3b59 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.736926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:43.738117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.738912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.739301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.739703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.740506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.742593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d3f12-0aef-4b0c-93ad-89ebc8564e7e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.805892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1034/2049] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:43 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:43.807524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.807817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.808093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.835165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:43.836231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.842519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459f7c5ac99a41678a2363b0972ae2ee', 'request_id': 'req-5c280a25-b6ac-46ac-9a8c-09972e31e581'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.848721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459f7c5ac99a41678a2363b0972ae2ee', 'request_id': 'req-5c280a25-b6ac-46ac-9a8c-09972e31e581'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.897050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:43.914538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1035/2050] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:43.915402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.923439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459f7c5ac99a41678a2363b0972ae2ee', 'request_id': 'req-5c280a25-b6ac-46ac-9a8c-09972e31e581'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.932147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] The token cache is being invalidate because project 459f7c5ac99a41678a2363b0972ae2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:43.932718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.932849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54be50-3501-4047-bb02-7a15e38cafd1 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.934171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:43.937373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1016/2051] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:43 2022] DELETE /identity/v3/projects/459f7c5ac99a41678a2363b0972ae2ee => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:43.938408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.940327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.941594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.942456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.942565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.942823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.943701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.945464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.957201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1036/2052] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:43.957914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.959699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.961087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c280a25-b6ac-46ac-9a8c-09972e31e581 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.967071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.968321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.968791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.969336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.970467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.972159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b48e4b-0d14-4114-9bdc-5c7c49147183 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.984337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1017/2053] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:43.986078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.986582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.987333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.011168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:44.013276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.033265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1018/2054] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:44.033956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.035618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.037127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31704c3f-9a76-455a-bd20-c435457bbd82 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.043792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.043964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.044254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.045054 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.046999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.049560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a87c8-395f-41c1-82b5-d6d9432897d5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.054339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.055302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.055741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.056020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] PATH_INFO: `/v3/projects/b831b92653e7451c9c5dfefdcd9cf048/users/9d953ce613954c5aaf49fa1dd8a78d82/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.058227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b831b92653e7451c9c5dfefdcd9cf048, user_id=9d953ce613954c5aaf49fa1dd8a78d82, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.063003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.066899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332ecd3-5ac4-4ac7-a8ce-b7c73a40e829 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.067009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.078570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1037/2055] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:44.079476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.094855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1019/2056] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:44 2022] PUT /identity/v3/projects/b831b92653e7451c9c5dfefdcd9cf048/users/9d953ce613954c5aaf49fa1dd8a78d82/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:44.096062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20279670-8ce4-4161-94ec-f04405818299 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.097099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac997ba-6729-4b87-9ae5-0631d0ee5298 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.097589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac997ba-6729-4b87-9ae5-0631d0ee5298 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.101401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac997ba-6729-4b87-9ae5-0631d0ee5298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.131674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0ac997ba-6729-4b87-9ae5-0631d0ee5298 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:44.133006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0ac997ba-6729-4b87-9ae5-0631d0ee5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.200938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.201957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.202316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.202667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] PATH_INFO: `/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25/users/2dd9c2c83e6047da8389b3f61d33ad04/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.226303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1020/2057] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:44.226303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.226303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.226303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.236165 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d8ce4ef7454b94b265e5378ab2bc25, user_id=2dd9c2c83e6047da8389b3f61d33ad04, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.238874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.262204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] MFA Rules not processed for user `9d953ce613954c5aaf49fa1dd8a78d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:44.263574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.331434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1038/2058] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:44 2022] PUT /identity/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25/users/2dd9c2c83e6047da8389b3f61d33ad04/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:44.332343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.387793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:44.390662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1021/2059] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:44.395743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.410501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:44.422622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3078e-09b4-4557-8a15-2e0d3a283765 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.481197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4a31e-cd81-4b3e-93c9-4c7f468cf796 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.491679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.493267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.493706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.494430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.495545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:44.501179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.501472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.533840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1039/2060] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:44.533840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.538686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.615012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.615973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.618943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.623216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.624146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.626482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.645924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1022/2061] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:44.646872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.670012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe54c-2978-46ba-816d-0da86ba5f1b3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.679275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.680426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.680426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.680640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.681202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.685001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.690506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aadc88-6939-4a86-97b5-0312780c89cb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.696559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.696559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.696559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.696559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.766424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.768206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.786462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1040/2062] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:44 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:44.791072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.791072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:44.810608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeff23-9250-4cfd-8962-9eb88a110612 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.821664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1023/2063] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:44 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:44.825496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.825496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.825496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.825898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] PATH_INFO: `/v3/projects/d09555e876d14361a8a7c6152de17a45` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.826997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.827152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09555e876d14361a8a7c6152de17a45)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.829127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.837456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:44.913343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e799f2-8856-4099-95ce-e1136f805cd9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:44.919832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:44.920934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:44.921258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:44.921528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] PATH_INFO: `/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25/users/2dd9c2c83e6047da8389b3f61d33ad04/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:44.923605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d8ce4ef7454b94b265e5378ab2bc25, user_id=2dd9c2c83e6047da8389b3f61d33ad04, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:44.926357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:44.937189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1024/2064] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:44 2022] PUT /identity/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25/users/2dd9c2c83e6047da8389b3f61d33ad04/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:44.937950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:44.994578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09555e876d14361a8a7c6152de17a45', 'request_id': 'req-358983e7-3c12-4b4b-ba77-80547f607a4f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.002984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09555e876d14361a8a7c6152de17a45', 'request_id': 'req-358983e7-3c12-4b4b-ba77-80547f607a4f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.007428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.027779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e02c-55d9-4e9c-b502-26c6d4a12660 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.083930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.084795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.085079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.085381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.086061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.087965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.095315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09555e876d14361a8a7c6152de17a45', 'request_id': 'req-358983e7-3c12-4b4b-ba77-80547f607a4f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.107622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] The token cache is being invalidate because project d09555e876d14361a8a7c6152de17a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:45.107909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.108556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-358983e7-3c12-4b4b-ba77-80547f607a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.110140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1041/2065] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:44 2022] DELETE /identity/v3/projects/d09555e876d14361a8a7c6152de17a45 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:45.111351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f35555-e05f-4923-8483-844698d13f90 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.111679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f35555-e05f-4923-8483-844698d13f90 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.111955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f35555-e05f-4923-8483-844698d13f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.144394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-40f35555-e05f-4923-8483-844698d13f90 None None] MFA Rules not processed for user `2dd9c2c83e6047da8389b3f61d33ad04`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:45.145801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-40f35555-e05f-4923-8483-844698d13f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.203881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1025/2066] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:44 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:45.203881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.204592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.229637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1042/2067] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:45.230361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f35555-e05f-4923-8483-844698d13f90 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.231708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-40f35555-e05f-4923-8483-844698d13f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.248781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f35555-e05f-4923-8483-844698d13f90 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.249957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.256845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.258176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.258403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.259011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.260081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.262974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.273909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb81c9-400e-4555-b156-407f6f47ebe3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.280063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1043/2068] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:45.280063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.281905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.284607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.285603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.286221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.287377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.289320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.295439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b9da-2bc2-4779-b272-48977bd9cd27 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.300512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.301743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.302032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.302313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.304428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.352412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.354123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.363320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1026/2069] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:45.366121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.367560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.370029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.378446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1044/2070] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:45.381830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d24ff5-d4dc-41a0-9db2-02d6071ffe04 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.387437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.388099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.388380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.388646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.389596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.391843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0c4a6-2e76-4be2-ba89-49ad0cd05cea admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.406324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1027/2071] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:45.407488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.418372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5748a3-284e-45b2-9c97-c15ab43d52b7 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.423318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.424372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.424656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.427002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] PATH_INFO: `/v3/projects/47cd27ad4219485cae0d8d85a2823948/users/92fed4b668e246f6b1b89a888c34f107/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.446415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47cd27ad4219485cae0d8d85a2823948, user_id=92fed4b668e246f6b1b89a888c34f107, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.449640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.525444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1045/2072] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:45 2022] PUT /identity/v3/projects/47cd27ad4219485cae0d8d85a2823948/users/92fed4b668e246f6b1b89a888c34f107/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:45.531161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.532875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.533175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.551021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:45.553244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.617312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1028/2073] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:45.628292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.633741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.675285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.694367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cd19e-149f-48ba-a114-240b74380466 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.698293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.698973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.699812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.700094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.709674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.711106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.721207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1046/2074] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:48.469065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:48.537850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4b40-e6d0-497b-a0bb-2dc87cca4eb8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:48.542029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:48.542799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:48.543086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:48.543421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:48.545802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:48.586850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:48.618072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:48.619850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4381a4c-6ea1-4751-9a84-606d9698c6af service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:48.632033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1029/2075] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:50.696267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.708602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61869ac-b341-4d6c-9d44-50d876460604 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.718875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.722256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.722396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.722720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.859955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.861630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:50.890108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1047/2076] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:51.505411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.505957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.506091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.535896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] MFA Rules not processed for user `92fed4b668e246f6b1b89a888c34f107`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:51.537117 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:51.608591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1030/2077] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:51.613247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:51.625777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe755a9-31f1-46df-8ca0-c7027b571689 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:51.630690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:51.631367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.631716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.632022 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.632757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:51.634997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:51.742843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1048/2078] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:51 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:51.748032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:51.759517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf2a72-29a1-4bec-bb8f-0c89c858a691 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:51.826535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:51.827249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.827537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.827815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.828520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:51.830555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:51.908050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1031/2079] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:51 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:51.912005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:51.924813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251215-412d-4a0c-a3f3-ac74848f0b64 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:51.930198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:51.942062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.942372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.942649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.943408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:51.945111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:51.956682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1049/2080] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:51.961296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:51.972112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6609-0789-498e-91d3-916613534186 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:51.977296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:51.978330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.978610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.978888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] PATH_INFO: `/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335/users/d66e80586c0747aabe2e4e9a0cf8db86/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.999881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a850cf48283f46bbac5b5a8e4f1a8335, user_id=d66e80586c0747aabe2e4e9a0cf8db86, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:52.002648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:52.089806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1032/2081] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:51 2022] PUT /identity/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335/users/d66e80586c0747aabe2e4e9a0cf8db86/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:52.094062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:52.105586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4e674-d7ee-4df3-812b-6d5306431bd7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:52.166231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:52.166894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:52.167176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:52.167956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:52.168687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:52.170413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:52.180282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1050/2082] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:52.185258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:52.195609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1be0c-5c16-4ec2-9735-e1e97230ff92 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:52.200481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:52.201429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:52.201763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:52.202043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] PATH_INFO: `/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335/users/d66e80586c0747aabe2e4e9a0cf8db86/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:52.204064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a850cf48283f46bbac5b5a8e4f1a8335, user_id=d66e80586c0747aabe2e4e9a0cf8db86, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:52.206866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3af3e-2b98-4886-b593-5ca8f6cbf691 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:52.217877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1033/2083] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:52 2022] PUT /identity/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335/users/d66e80586c0747aabe2e4e9a0cf8db86/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:55.840623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:55.843271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:55.857757 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7a628-0a5d-4175-9797-c1cfe986a1dd admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:55.924367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:55.960805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:55.961795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:55.967936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:55.967936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] PATH_INFO: `/v3/users/3c44d6f249d94133a2b48da40efb08a9` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:55.967936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c44d6f249d94133a2b48da40efb08a9)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:55.967936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:56.269763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c44d6f249d94133a2b48da40efb08a9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:56.270572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c44d6f249d94133a2b48da40efb08a9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:56.271212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c44d6f249d94133a2b48da40efb08a9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:56.281726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c44d6f249d94133a2b48da40efb08a9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:56.308850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c44d6f249d94133a2b48da40efb08a9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:56.404747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1051/2084] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:55 2022] DELETE /identity/v3/users/3c44d6f249d94133a2b48da40efb08a9 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:56.410034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:56.410846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:56.411280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:56.464228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:56.471967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:56.491738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:56.600627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1034/2085] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:56.609747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:56.612111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:56.692253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55ccf-96c2-44be-839d-3ddd49d451e1 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:56.697195 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:56.698171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:56.698909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:56.699396 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:56.711338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:56.712983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:56.744176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1052/2086] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:57.352281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:57.353768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:57.372967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0c2a8-8e60-41a9-a47a-c181ad3ad73a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:57.385418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:57.387472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:57.387773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:57.388183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] PATH_INFO: `/v3/projects/ee630025d32e4b36b07704a51b7ea0ea` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:57.392087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee630025d32e4b36b07704a51b7ea0ea)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:57.394427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:57.607459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee630025d32e4b36b07704a51b7ea0ea', 'request_id': 'req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:57.619575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee630025d32e4b36b07704a51b7ea0ea', 'request_id': 'req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:57.695671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee630025d32e4b36b07704a51b7ea0ea', 'request_id': 'req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:57.709668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] The token cache is being invalidate because project ee630025d32e4b36b07704a51b7ea0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:57.710834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:57.711624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:57.717754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:57.724100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1035/2087] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:57 2022] DELETE /identity/v3/projects/ee630025d32e4b36b07704a51b7ea0ea => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:57.731641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c9b7-9a9f-48b4-bc45-5cc8774da4ee service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:57.815556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:57.816796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:57.818855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:57.819276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] PATH_INFO: `/v3/users/bbb18368bcde4526a58ed68dccbe37b2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:57.821439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb18368bcde4526a58ed68dccbe37b2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:57.823976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:57.830011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:58.070931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb18368bcde4526a58ed68dccbe37b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:58.071620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb18368bcde4526a58ed68dccbe37b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:58.072435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb18368bcde4526a58ed68dccbe37b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:58.079635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb18368bcde4526a58ed68dccbe37b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:58.107239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb18368bcde4526a58ed68dccbe37b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:58.227635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1053/2088] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:57 2022] DELETE /identity/v3/users/bbb18368bcde4526a58ed68dccbe37b2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:58.920585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:58.924310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:58.949414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c3cbc-9253-4a68-9b3e-2d89a4e5cc6c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:59.115650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:59.116710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:59.117008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:59.117416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] PATH_INFO: `/v3/projects/ef03beaddb594300b584f2382f35ac65` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:59.120241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef03beaddb594300b584f2382f35ac65)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:59.122708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:59.218100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:59.226864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef03beaddb594300b584f2382f35ac65', 'request_id': 'req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.235174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef03beaddb594300b584f2382f35ac65', 'request_id': 'req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.317699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef03beaddb594300b584f2382f35ac65', 'request_id': 'req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.344049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] The token cache is being invalidate because project ef03beaddb594300b584f2382f35ac65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:59.344231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.349567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63a5823d-f5c2-4ea9-a68b-008a93ed18c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.349567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1036/2089] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:58 2022] DELETE /identity/v3/projects/ef03beaddb594300b584f2382f35ac65 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:59.351662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:59.354233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:59.369689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e4f5a-4e77-4b34-aab3-fa383a0e9c47 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:59.452701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:59.453806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:59.454292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:59.454720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] PATH_INFO: `/v3/users/2dd9c2c83e6047da8389b3f61d33ad04` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:59.456168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd9c2c83e6047da8389b3f61d33ad04)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:59.458426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:59.676674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd9c2c83e6047da8389b3f61d33ad04'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.677577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd9c2c83e6047da8389b3f61d33ad04'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.682447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd9c2c83e6047da8389b3f61d33ad04'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.701820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd9c2c83e6047da8389b3f61d33ad04'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.723107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd9c2c83e6047da8389b3f61d33ad04'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:59.734703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:59.735063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:59.735416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:59.787518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] MFA Rules not processed for user `d66e80586c0747aabe2e4e9a0cf8db86`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:59.788893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:59.794552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:59.799788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1054/2090] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:59 2022] DELETE /identity/v3/users/2dd9c2c83e6047da8389b3f61d33ad04 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:59.947302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1037/2091] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:59.962107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:59.963656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.048189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e5996-ac3a-4637-bfb8-2261da1e7f3c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:00.051823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:00.052477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.053202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.053202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.054933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.121009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:00.123014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd79bad-99bf-4542-ac54-1082e8bae653 service glance] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:00.134808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1055/2092] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:00.319743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:00.324476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.344628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc62a0-72c2-4bd0-a4ed-91375cb38da4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:00.400086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.400599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.400693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.419140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:00.419779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:00.420928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.421038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.421539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.422568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] PATH_INFO: `/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.424674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d8ce4ef7454b94b265e5378ab2bc25)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:00.426935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:00.504744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1056/2093] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:00.564079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:00.565545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.576488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d8ce4ef7454b94b265e5378ab2bc25', 'request_id': 'req-863db954-db35-4666-889f-0ed720f94ca0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:00.581308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:00.584167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d8ce4ef7454b94b265e5378ab2bc25', 'request_id': 'req-863db954-db35-4666-889f-0ed720f94ca0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:00.620564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:00.650901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab412d28-f060-46b5-bcbe-31a497081bdc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:00.660768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d8ce4ef7454b94b265e5378ab2bc25', 'request_id': 'req-863db954-db35-4666-889f-0ed720f94ca0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:00.674885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] The token cache is being invalidate because project b5d8ce4ef7454b94b265e5378ab2bc25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:00.675377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:00.676043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-863db954-db35-4666-889f-0ed720f94ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:00.681109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1038/2094] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:00 2022] DELETE /identity/v3/projects/b5d8ce4ef7454b94b265e5378ab2bc25 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:00.683257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a5de-81a6-4e35-9ec2-876b4d365969 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.684209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a5de-81a6-4e35-9ec2-876b4d365969 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.684580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a5de-81a6-4e35-9ec2-876b4d365969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.705789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9980a5de-81a6-4e35-9ec2-876b4d365969 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:00.707122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9980a5de-81a6-4e35-9ec2-876b4d365969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.724341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:00.725559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.725860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.730479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.730479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.779611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1039/2095] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:00.787631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.787935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.788323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.798600 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:00.799739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:00.809586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1057/2096] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:41:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:41:00.810831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:00.812589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:00.835656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1040/2097] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:00.836464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:00.871024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790527-7a81-49d5-a9c2-c7538b433502 service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:00.878163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:00.886230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:00.911965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:00.918984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1058/2098] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:41:00.925506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99401-c320-4218-ad23-06a4fe775715 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:00.934104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:00.935153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:00.935658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:00.942741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:00.942741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:00.942741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:01.074284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1041/2099] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:00 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:01.134917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:01.134917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2776e-21a9-41a6-be1a-f8ae2c0730a8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:01.210833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:01.212345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:01.213071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:01.214253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:01.215776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:01.218347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:01.421746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:01.431596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1059/2100] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:01 2022] POST /identity/v3/users => generated 292 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:01.434601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:01.449180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7490b-50df-4d08-be60-81affe6986a8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:01.457675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:01.461976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:01.462273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:01.462603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:01.463356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:01.466994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:01.482342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1042/2101] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:01.486296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:01.498954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a322148-65fd-40f5-835d-4c0ef67d8e1b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:01.505503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:01.506401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:01.507084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:01.507556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:01.508722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:01.510824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7cf3f-169a-4788-a796-ce8f85d3451f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:01.524110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1060/2102] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:01.528965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:01.542915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2945196-311a-4098-80be-b29fcf003bd0 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:01.555947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:01.557055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:01.557512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:01.557891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] PATH_INFO: `/v3/projects/895b9837399943edab5ad28e3c7abce5/users/fbfdd2fcb7db4a7ebaff1f3be11d3222/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:01.576570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895b9837399943edab5ad28e3c7abce5, user_id=fbfdd2fcb7db4a7ebaff1f3be11d3222, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:01.581531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:01.726456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1043/2103] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:01 2022] PUT /identity/v3/projects/895b9837399943edab5ad28e3c7abce5/users/fbfdd2fcb7db4a7ebaff1f3be11d3222/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:01.727339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:01.727632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:01.728959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:01.764406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] MFA Rules not processed for user `fbfdd2fcb7db4a7ebaff1f3be11d3222`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:01.765671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:01.892352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1061/2104] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:01.899899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:01.903122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:01.995869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a86494-ac7f-41ae-80a2-4f378d031af3 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:02.000455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:02.001444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:02.002199 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:02.002694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:02.015931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:02.026796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c07d93-4289-4220-a8b1-97c8724b0650 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:02.032827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1044/2105] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:06.070218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:06.233391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fe3b4-9c6d-4632-97ad-f6a8f6e36a8a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:06.233391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:06.233391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.233391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.234992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] PATH_INFO: `/v3/users/fbfdd2fcb7db4a7ebaff1f3be11d3222` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.234992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfdd2fcb7db4a7ebaff1f3be11d3222)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:06.234992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:06.768665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfdd2fcb7db4a7ebaff1f3be11d3222'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:06.773955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfdd2fcb7db4a7ebaff1f3be11d3222'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:06.773955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfdd2fcb7db4a7ebaff1f3be11d3222'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:06.789067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfdd2fcb7db4a7ebaff1f3be11d3222'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:06.791687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18808b3-94f6-4924-a46f-e4ab93387e50 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.792240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18808b3-94f6-4924-a46f-e4ab93387e50 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.792332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18808b3-94f6-4924-a46f-e4ab93387e50 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.794054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1045/2106] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:41:06.800608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecdcb-8065-47f8-a472-775f7bb6e6da None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.800895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecdcb-8065-47f8-a472-775f7bb6e6da None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.801188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecdcb-8065-47f8-a472-775f7bb6e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.827894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfdd2fcb7db4a7ebaff1f3be11d3222'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:06.830679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-465ecdcb-8065-47f8-a472-775f7bb6e6da None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:06.835263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-465ecdcb-8065-47f8-a472-775f7bb6e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:06.853840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:06.914343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:06.922620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1062/2107] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:06 2022] DELETE /identity/v3/users/fbfdd2fcb7db4a7ebaff1f3be11d3222 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:06.923692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:06.934581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:06.947525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1046/2108] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:06.950016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.950675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.951167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.987577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:06.988564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:07.100722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1047/2109] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:07.101732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:07.103132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:07.107322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e1431-1de3-466c-814f-1d961bb0d577 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:07.122868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:07.123682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:07.123957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:07.124231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] PATH_INFO: `/v3/users/83e06ab7cd924e07a79efaafa078a174` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:07.126224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e06ab7cd924e07a79efaafa078a174)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:07.132225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:07.231531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c699-7b67-4659-a36a-3790f21a8152 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:07.237317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:07.238111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:07.238496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:07.239007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:07.249763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:07.251504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:07.262412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1048/2110] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:07.271404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:07.275032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:07.414798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:07.432808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e06ab7cd924e07a79efaafa078a174'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:07.438401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e06ab7cd924e07a79efaafa078a174'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:07.438401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e06ab7cd924e07a79efaafa078a174'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:07.451324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e7667-1ab1-450c-b33d-80c53ad53c71 service placement] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:07.453685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e06ab7cd924e07a79efaafa078a174'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:07.461269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:07.462960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:07.468203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:07.472609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:07.473352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e06ab7cd924e07a79efaafa078a174'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:07.474522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:07.496634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:07.498132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:07.510676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1049/2111] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:07.559306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1063/2112] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:06 2022] DELETE /identity/v3/users/83e06ab7cd924e07a79efaafa078a174 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:08.091053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:08.093343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:08.115734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334e9d8-8560-4072-8a59-716f1bdc7405 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:08.226158 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:08.227564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.227856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.228137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] PATH_INFO: `/v3/projects/165d7892df2f450d927c92d71cd96f29` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.230078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165d7892df2f450d927c92d71cd96f29)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:08.232519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:08.341674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:08.343952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:08.358166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ee8d-501d-4cb3-8aae-8ab5c1b1b0d0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:08.395923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165d7892df2f450d927c92d71cd96f29', 'request_id': 'req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.415973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165d7892df2f450d927c92d71cd96f29', 'request_id': 'req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.434913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:08.435800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.436082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.436358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] PATH_INFO: `/v3/projects/895b9837399943edab5ad28e3c7abce5` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.437602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=895b9837399943edab5ad28e3c7abce5)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:08.442371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:08.491387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:08.505246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165d7892df2f450d927c92d71cd96f29', 'request_id': 'req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.524796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] The token cache is being invalidate because project 165d7892df2f450d927c92d71cd96f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:08.525148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.525968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fc6fe21b-d06b-43aa-97e9-73e9b61339a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.528026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1050/2113] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:08 2022] DELETE /identity/v3/projects/165d7892df2f450d927c92d71cd96f29 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:08.533231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f623c4-1445-433c-b929-28b717535d87 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.533522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f623c4-1445-433c-b929-28b717535d87 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.533799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f623c4-1445-433c-b929-28b717535d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.561019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-30f623c4-1445-433c-b929-28b717535d87 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:08.562985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-30f623c4-1445-433c-b929-28b717535d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:08.585187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1051/2114] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:08.586137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.586432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.586712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.617637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:08.619190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:08.629131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895b9837399943edab5ad28e3c7abce5', 'request_id': 'req-273f92c1-8d50-450f-9449-359771b94e3e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.643612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895b9837399943edab5ad28e3c7abce5', 'request_id': 'req-273f92c1-8d50-450f-9449-359771b94e3e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.712277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895b9837399943edab5ad28e3c7abce5', 'request_id': 'req-273f92c1-8d50-450f-9449-359771b94e3e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.726017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] The token cache is being invalidate because project 895b9837399943edab5ad28e3c7abce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:08.726314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.727905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:08.730426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1064/2115] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:08 2022] DELETE /identity/v3/projects/895b9837399943edab5ad28e3c7abce5 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:08.752188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1052/2116] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:08.756290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:08.770028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f92c1-8d50-450f-9449-359771b94e3e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:08.776154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:08.776796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.777089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.777417 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.778252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:08.778558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:08.780222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:08.801558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1065/2117] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:41:08.802852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:08.815082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53433a-93f9-46f8-bc1b-efe561d2760e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:08.819803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:08.820552 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:08.820889 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:08.821218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:08.822291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:08.825462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:08.841461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:08.950867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:08.956251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1053/2118] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:08 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:41:08.961695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fdb7b-528d-4e5e-afec-8a1e1c2ad612 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.029538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.030484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.030769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.031049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.031762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.033697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.143201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.145979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:09.148103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1066/2119] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:08 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:09.151868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.169634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3def2-4213-4982-a5e2-486f0f7165f0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.172498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9ef9a-e856-4989-995a-d5bf9493a1e8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.175308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.176078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.176350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.176488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.176786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.177306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.177598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.177714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.177878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] PATH_INFO: `/v3/users/93057a5b9b4d4b87bb960e1a776ac61c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.179207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93057a5b9b4d4b87bb960e1a776ac61c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.179710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.183868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.196343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1067/2120] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:09.200727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.219989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190a7b-4f1d-4d13-89aa-5dcda9716ae5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.225549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.226302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.226629 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.226972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.227850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.230168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.242422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1068/2121] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:09.247175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.258209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730fc8d-f790-49bc-b30b-4ff7c7bfaefb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.263385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.264695 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.265138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.265594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] PATH_INFO: `/v3/projects/de5fc0bb321b4d02a13fadc91a7b7e52/users/a26b10a93dc7436bad2ce5a8e9b16d04/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.282640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5fc0bb321b4d02a13fadc91a7b7e52, user_id=a26b10a93dc7436bad2ce5a8e9b16d04, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.285360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1da7c9-383e-46ff-a075-78ad1f24d62f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.361472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1069/2122] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:09 2022] PUT /identity/v3/projects/de5fc0bb321b4d02a13fadc91a7b7e52/users/a26b10a93dc7436bad2ce5a8e9b16d04/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:09.372457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.372749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.373032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.402240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93057a5b9b4d4b87bb960e1a776ac61c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:09.402968 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93057a5b9b4d4b87bb960e1a776ac61c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:09.403814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93057a5b9b4d4b87bb960e1a776ac61c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:09.406855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] MFA Rules not processed for user `a26b10a93dc7436bad2ce5a8e9b16d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:09.408265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:09.415810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93057a5b9b4d4b87bb960e1a776ac61c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:09.434935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cf820652-aeee-4893-aec9-3bbbb6406907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93057a5b9b4d4b87bb960e1a776ac61c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:09.440646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:09.488616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1070/2123] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:09.492887 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.494423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:09.513128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1054/2124] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:09 2022] DELETE /identity/v3/users/93057a5b9b4d4b87bb960e1a776ac61c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:09.518738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.519029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.519309 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.538412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:09.539736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:09.581945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc750130-e3e7-4b00-9b8c-383b0f19b274 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.613193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1055/2125] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:09.625261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.627130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:09.665985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.666821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.667420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.668657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.669411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.671529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.699911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b206-224f-4ebe-a3a3-c1401cf695d7 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.704295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.704807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.705151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.705520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.719233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.720936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:09.731509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1056/2126] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:09.814304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1071/2127] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:09 2022] POST /identity/v3/projects => generated 375 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:41:09.818022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:09.831857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37586d59-0e11-4e86-bd10-ac6f93bfe5e1 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:09.848907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:09.935824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:09.946587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:09.947194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:09.947732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:09.948709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:09.951574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.073069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1057/2128] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:09 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:10.074153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.085790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb67c0-4542-4f48-9754-3e7d4d9e0f4e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.090455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.091204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.091489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.091805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.092761 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.094439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.107689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1072/2129] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:10.111817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.135318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c878d-5c0e-459e-a110-f9769a3bdf45 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.140942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.141924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.142213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.142494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] PATH_INFO: `/v3/projects/d03e2d8fdde446e9a780a46a4f353403/users/7646afb7b6f7474698b3dad8787636f9/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.163859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03e2d8fdde446e9a780a46a4f353403, user_id=7646afb7b6f7474698b3dad8787636f9, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.167913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.217271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.220630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:10.235755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0f7c4-b5f8-4e6b-bd74-4691b977e458 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.240805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.241699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.241982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.242280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] PATH_INFO: `/v3/projects/220a5d3b89254edfa065fc9622521866` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.243474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220a5d3b89254edfa065fc9622521866)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.246458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.260867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1058/2130] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:10 2022] PUT /identity/v3/projects/d03e2d8fdde446e9a780a46a4f353403/users/7646afb7b6f7474698b3dad8787636f9/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:10.297084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.363713 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c4974-591c-48d9-a769-c8f358d1d613 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.368505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.369773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.370061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.370339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.371333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.373375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.384287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1059/2131] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:10.389356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.398913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc08e6-1ed4-48ea-bb9f-548a19dd1a87 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.403453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.404302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.404582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.404854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] PATH_INFO: `/v3/projects/d03e2d8fdde446e9a780a46a4f353403/users/7646afb7b6f7474698b3dad8787636f9/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.406831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03e2d8fdde446e9a780a46a4f353403, user_id=7646afb7b6f7474698b3dad8787636f9, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.409470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043efb2-f71d-4a2f-ac3c-4111b00708bf admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.441068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1060/2132] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:10 2022] PUT /identity/v3/projects/d03e2d8fdde446e9a780a46a4f353403/users/7646afb7b6f7474698b3dad8787636f9/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:10.451753 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3591-6448-405d-a83a-e761ef375045 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.452054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3591-6448-405d-a83a-e761ef375045 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.452334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b3591-6448-405d-a83a-e761ef375045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.478134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-541b3591-6448-405d-a83a-e761ef375045 None None] MFA Rules not processed for user `7646afb7b6f7474698b3dad8787636f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:10.479344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-541b3591-6448-405d-a83a-e761ef375045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:10.525057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:10.547990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1061/2133] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:10.561278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b3591-6448-405d-a83a-e761ef375045 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.563615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-541b3591-6448-405d-a83a-e761ef375045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:10.632618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b3591-6448-405d-a83a-e761ef375045 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.636864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.637540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.637825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.638103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.691092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:10.695335 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220a5d3b89254edfa065fc9622521866', 'request_id': 'req-7334b29f-fe7a-4807-9ce7-af6a56d382d6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:10.696015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.697500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:10.701593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220a5d3b89254edfa065fc9622521866', 'request_id': 'req-7334b29f-fe7a-4807-9ce7-af6a56d382d6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:10.706963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1062/2134] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:41:10.768217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220a5d3b89254edfa065fc9622521866', 'request_id': 'req-7334b29f-fe7a-4807-9ce7-af6a56d382d6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:10.785236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] The token cache is being invalidate because project 220a5d3b89254edfa065fc9622521866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:10.785781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:10.786453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:10.790949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1073/2135] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:10 2022] DELETE /identity/v3/projects/220a5d3b89254edfa065fc9622521866 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:10.792908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:10.865263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e8d7-67d9-474c-a758-6ed903e02996 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:10.919244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:10.935194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:10.936342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:10.936909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:10.937367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] PATH_INFO: `/v3/users/9d953ce613954c5aaf49fa1dd8a78d82` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:10.938787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d953ce613954c5aaf49fa1dd8a78d82)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:10.940778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:11.163026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d953ce613954c5aaf49fa1dd8a78d82'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:11.163834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d953ce613954c5aaf49fa1dd8a78d82'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:11.164509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d953ce613954c5aaf49fa1dd8a78d82'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:11.172217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d953ce613954c5aaf49fa1dd8a78d82'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:11.187171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-511a8be2-b8a5-412c-a0b7-966e1258f26f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d953ce613954c5aaf49fa1dd8a78d82'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:11.282463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1063/2136] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:10 2022] DELETE /identity/v3/users/9d953ce613954c5aaf49fa1dd8a78d82 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:11.822483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:11.824027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:11.848833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334b29f-fe7a-4807-9ce7-af6a56d382d6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:11.922172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:11.923480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:11.923872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:11.924255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] PATH_INFO: `/v3/projects/b831b92653e7451c9c5dfefdcd9cf048` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:11.925610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b831b92653e7451c9c5dfefdcd9cf048)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:11.927640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.024142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b831b92653e7451c9c5dfefdcd9cf048', 'request_id': 'req-723670dd-8ae1-4e47-a667-85142b02ac35'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:12.030152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b831b92653e7451c9c5dfefdcd9cf048', 'request_id': 'req-723670dd-8ae1-4e47-a667-85142b02ac35'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:12.082842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b831b92653e7451c9c5dfefdcd9cf048', 'request_id': 'req-723670dd-8ae1-4e47-a667-85142b02ac35'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:12.098127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] The token cache is being invalidate because project b831b92653e7451c9c5dfefdcd9cf048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:12.102002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:12.102002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-723670dd-8ae1-4e47-a667-85142b02ac35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:12.102002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1074/2137] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:11 2022] DELETE /identity/v3/projects/b831b92653e7451c9c5dfefdcd9cf048 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:12.105924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.106383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.106792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.126190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:12.127650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:12.198919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1064/2138] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:12.210860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.211184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.211446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.247096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:12.249186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:12.269469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1075/2139] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:12.270557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.295096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885068-ecdd-44fa-90b3-be2718e139fc None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.301597 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.303546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.304023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.304476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.305466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:12.306001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.308431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.324843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1065/2140] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:41:12.329571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.343595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac63bdc-f407-4010-b26f-c1c5c9237894 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.349254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.349923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.350210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.350487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.351344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.354180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.480419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1076/2141] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:12 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:41:12.484604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.497814 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13ce6d-0f16-4b4d-8358-ab3d7393abe7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.567509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:12.582141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.585185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.585185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.585185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.585185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.586805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.678512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1066/2142] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:12 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:12.690089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.704707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd5241-8c02-421e-8124-1f6884e8eb03 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.709595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.710517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.710946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.711276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.712012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.713659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.727528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1077/2143] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:12.731279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.742030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65703bfd-f220-43c0-afb9-4c45279dad2b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.748318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.749174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.749674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.750051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.750885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.752851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7f528-7107-466f-b555-1c3b1960fdeb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.767147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1067/2144] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:12.771189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:12.782280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29cbe-a805-4f32-8319-fbb945867099 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:12.787387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:12.788385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.788736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.789083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] PATH_INFO: `/v3/projects/5286480de5f6455d913524d4b9079563/users/7feb90c0fdba4f4ab7427a77efd94f98/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.807918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5286480de5f6455d913524d4b9079563, user_id=7feb90c0fdba4f4ab7427a77efd94f98, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:12.812773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:12.885319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:12.891439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1078/2145] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:12 2022] PUT /identity/v3/projects/5286480de5f6455d913524d4b9079563/users/7feb90c0fdba4f4ab7427a77efd94f98/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:12.908769 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:12.909103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:12.909357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:12.949093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] MFA Rules not processed for user `7feb90c0fdba4f4ab7427a77efd94f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:12.950346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:13.031044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1068/2146] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:13.031667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.111802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008cc0a-52c1-4068-aec7-1da078804e47 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.116717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.117578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.117866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.118148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.119017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.121813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.230437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1079/2147] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:13 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:13.234885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.247426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef5e2b-d619-411c-a028-d85d3a55e565 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.328669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.329392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.329680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.329967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.338919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.338919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.407179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1069/2148] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:13 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:13.408511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.421876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb65f91-bfa0-4178-9de2-4eaf9f365e1d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.427087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.427884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.428313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.428596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.429650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.432270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.444499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1080/2149] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:13.448586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.478084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca816-3ca0-4d8c-8b88-bd507396d914 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.484423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.485728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.486231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.486978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] PATH_INFO: `/v3/projects/5160320a0fec4dadb0d4c290df758f28/users/09c1b63d96664807b187a24952d14af7/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.506416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5160320a0fec4dadb0d4c290df758f28, user_id=09c1b63d96664807b187a24952d14af7, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.509686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.588280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1070/2150] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:13 2022] PUT /identity/v3/projects/5160320a0fec4dadb0d4c290df758f28/users/09c1b63d96664807b187a24952d14af7/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:13.643502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.660434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe23e62-feaa-4e6b-b29b-a9c57a93ec69 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.724853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.725870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.726155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.726432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.727177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.728831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e67d2-38e4-4d41-94d1-624049ebc332 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.739967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1081/2151] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:13.745348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.757243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f27ce-5724-4da9-86af-c9e1fc96b9a9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.762442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.763324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.763614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.763890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] PATH_INFO: `/v3/projects/5160320a0fec4dadb0d4c290df758f28/users/09c1b63d96664807b187a24952d14af7/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.765898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5160320a0fec4dadb0d4c290df758f28, user_id=09c1b63d96664807b187a24952d14af7, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:13.768522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:13.780204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1071/2152] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:13 2022] PUT /identity/v3/projects/5160320a0fec4dadb0d4c290df758f28/users/09c1b63d96664807b187a24952d14af7/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:13.802878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaedd18-15c9-40a6-a603-621eafda931e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.803197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaedd18-15c9-40a6-a603-621eafda931e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.803478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaedd18-15c9-40a6-a603-621eafda931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:13.836192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4eaedd18-15c9-40a6-a603-621eafda931e None None] MFA Rules not processed for user `09c1b63d96664807b187a24952d14af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:13.837434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4eaedd18-15c9-40a6-a603-621eafda931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:13.906434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:13.916324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1082/2153] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:13.927440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:13.929137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:13.942603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:13.989930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e722-4bab-45fe-9e4d-3d10872cab8c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:13.993652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:13.994235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:13.994522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:13.994799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:14.004004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:14.005453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:14.015844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1072/2154] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:15.160521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:15.161031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:15.161174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:15.179267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] MFA Rules not processed for user `92fed4b668e246f6b1b89a888c34f107`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:15.180436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:15.238009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1083/2155] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:16.881181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:16.946647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73367a0b-c6c3-4038-8e44-147201764709 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:16.952492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:16.953399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:16.953688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:16.953964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] PATH_INFO: `/v3/users/7feb90c0fdba4f4ab7427a77efd94f98` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:16.965793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7feb90c0fdba4f4ab7427a77efd94f98)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:16.965793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:17.202997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:17.213105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b248-1e8f-45ce-91da-feaad77eba9f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:17.216963 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:17.217640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:17.217931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:17.218210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:17.219992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:17.239806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7feb90c0fdba4f4ab7427a77efd94f98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:17.240548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7feb90c0fdba4f4ab7427a77efd94f98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:17.241182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7feb90c0fdba4f4ab7427a77efd94f98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:17.249079 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7feb90c0fdba4f4ab7427a77efd94f98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:17.262134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7feb90c0fdba4f4ab7427a77efd94f98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:17.284222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:17.285836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f61c5-8d9b-4f2f-b503-d2560c1f39c6 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:17.295738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1084/2156] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:41:17.344983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1073/2157] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:16 2022] DELETE /identity/v3/users/7feb90c0fdba4f4ab7427a77efd94f98 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:17.349535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:17.349829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:17.350112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:17.366849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:17.371345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:17.430754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1085/2158] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:17.437922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:17.439618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:17.480222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:17.502770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f357e-319c-403a-8c2c-76ee12773193 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:17.506410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:17.507049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:17.507332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:17.507607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:17.517847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:17.520061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:17.529326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1074/2159] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:17.998739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:18.000211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:18.014768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207dc1b-06ca-4256-a399-084976b21ea9 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:18.021800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:18.022773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:18.023106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:18.023329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] PATH_INFO: `/v3/projects/5286480de5f6455d913524d4b9079563` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:18.024583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5286480de5f6455d913524d4b9079563)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:18.026561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:18.124679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:18.141616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5286480de5f6455d913524d4b9079563', 'request_id': 'req-93899925-4c0e-4433-aaa9-0b67cb45966f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.157480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5286480de5f6455d913524d4b9079563', 'request_id': 'req-93899925-4c0e-4433-aaa9-0b67cb45966f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.253902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5286480de5f6455d913524d4b9079563', 'request_id': 'req-93899925-4c0e-4433-aaa9-0b67cb45966f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.266183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] The token cache is being invalidate because project 5286480de5f6455d913524d4b9079563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:18.266461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.267112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.268704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1086/2160] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:17 2022] DELETE /identity/v3/projects/5286480de5f6455d913524d4b9079563 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:18.272802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:18.283224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6ca12-0175-4ced-bef8-2d53de9db73c service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:18.348067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:18.349061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:18.349369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:18.349643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] PATH_INFO: `/v3/users/09c1b63d96664807b187a24952d14af7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:18.350827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c1b63d96664807b187a24952d14af7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:18.352739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:18.507333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c1b63d96664807b187a24952d14af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.507920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c1b63d96664807b187a24952d14af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.508527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c1b63d96664807b187a24952d14af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.516179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c1b63d96664807b187a24952d14af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.528303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54e46ef2-9739-42f6-b9a0-d91972a7cb6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c1b63d96664807b187a24952d14af7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:18.614246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1075/2161] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:18 2022] DELETE /identity/v3/users/09c1b63d96664807b187a24952d14af7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:19.031777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.033318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:19.050480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899925-4c0e-4433-aaa9-0b67cb45966f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.109151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.110196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.110478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.110790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] PATH_INFO: `/v3/projects/5160320a0fec4dadb0d4c290df758f28` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.112066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5160320a0fec4dadb0d4c290df758f28)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.114018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.196981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5160320a0fec4dadb0d4c290df758f28', 'request_id': 'req-96abceed-7c88-4a63-8bd3-52c261577e9a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:19.202488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5160320a0fec4dadb0d4c290df758f28', 'request_id': 'req-96abceed-7c88-4a63-8bd3-52c261577e9a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:19.259347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5160320a0fec4dadb0d4c290df758f28', 'request_id': 'req-96abceed-7c88-4a63-8bd3-52c261577e9a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:19.270970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] The token cache is being invalidate because project 5160320a0fec4dadb0d4c290df758f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:19.271259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:19.271897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-96abceed-7c88-4a63-8bd3-52c261577e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:19.273470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1087/2162] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:19 2022] DELETE /identity/v3/projects/5160320a0fec4dadb0d4c290df758f28 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:19.278724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.279143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.279410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.297137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:19.298295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:19.347994 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:19.360425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1076/2163] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:19.367899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.368200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.368508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.395873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:19.397706 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:19.408325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1088/2164] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:19.413606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.423665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2d7d-4f29-4684-8318-25afd3aef368 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.433756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.434432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.434720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.435009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.435747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:19.436702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.439788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.453042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1077/2165] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:41:19.457434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.466744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c825c-c9ac-4caf-8491-2bd91790e4b4 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.471525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.472194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.472604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.472900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.473739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.475796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.562888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1089/2166] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:19 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:41:19.566721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.577279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ec4c4-5351-451f-a1fe-92638557290b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.631403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.632060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.632341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.632618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.633330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.635877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.734501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1078/2167] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:19 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:41:19.738620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.752504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e2a74-8e46-4cc9-ba18-89a20fcda966 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.761390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.762052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.762331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.762605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.763329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.764912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.776462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1090/2168] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:19.780334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.790409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ddfdb-6d88-4e10-b12b-b709a48768ef admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.795089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.795743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.796030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.796307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.797032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.798646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4e08b-fe0e-40a9-858f-69f36f926dbd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.808334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1079/2169] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:19.812183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:19.822496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f920a8f-9af1-40a2-a03d-3fd9bbab30c7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:19.827116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:19.827983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.828265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.828542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] PATH_INFO: `/v3/projects/e898ce340a7a4ac6af1639a02ebe9ef1/users/121c1b61547a482c93113de47152c742/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.830746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:19.843678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e898ce340a7a4ac6af1639a02ebe9ef1, user_id=121c1b61547a482c93113de47152c742, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:19.846367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:19.920946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1091/2170] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:19 2022] PUT /identity/v3/projects/e898ce340a7a4ac6af1639a02ebe9ef1/users/121c1b61547a482c93113de47152c742/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:19.932274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:19.932565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:19.932845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:19.959422 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] MFA Rules not processed for user `121c1b61547a482c93113de47152c742`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:19.960619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:20.023664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1080/2171] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:20.028773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.096755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed837-df29-4a88-ad49-024b827fd32b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.101363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.102014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.102298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.102575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.103286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.105209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.186429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1092/2172] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:20 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:20.191153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.202402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8162-27ec-4917-bb68-794ac5b7d873 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.247951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:20.257989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.258657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.258939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.259811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.260509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.262497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.364409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1081/2173] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:20 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:20.368526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.379381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0698e-89a4-4ec4-bb45-66546eb4b884 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.384029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.384707 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.384988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.385302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.386025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.387605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.398464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1093/2174] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:20.403213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.413307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d464f3-b1fe-491a-913b-5131023cd3b6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.418644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.419519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.419807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.420084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] PATH_INFO: `/v3/projects/b0851abc260146e0861207d4757e7fcf/users/f4b662f3de1b40259e42b0865943ac98/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.435355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0851abc260146e0861207d4757e7fcf, user_id=f4b662f3de1b40259e42b0865943ac98, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.438100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.505261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1082/2175] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:20 2022] PUT /identity/v3/projects/b0851abc260146e0861207d4757e7fcf/users/f4b662f3de1b40259e42b0865943ac98/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:20.509952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.519972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a807f5-e73b-4737-9f42-4426d617ddae admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.579903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.580571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.580853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.581126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.581884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.583474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0947a-3441-4338-b2fd-5df2d5a9b212 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.594680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1094/2176] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:41:20.598612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.609020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1267b-2efd-4c29-9813-4d793fe88a19 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.613652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.614598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.614828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.615441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] PATH_INFO: `/v3/projects/b0851abc260146e0861207d4757e7fcf/users/f4b662f3de1b40259e42b0865943ac98/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.617250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0851abc260146e0861207d4757e7fcf, user_id=f4b662f3de1b40259e42b0865943ac98, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.619917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.630825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1083/2177] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:20 2022] PUT /identity/v3/projects/b0851abc260146e0861207d4757e7fcf/users/f4b662f3de1b40259e42b0865943ac98/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:20.641397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef398b-bb19-4a15-a2c1-0bc7b3a80cf4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.641690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef398b-bb19-4a15-a2c1-0bc7b3a80cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.641968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef398b-bb19-4a15-a2c1-0bc7b3a80cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.668451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d8ef398b-bb19-4a15-a2c1-0bc7b3a80cf4 None None] MFA Rules not processed for user `f4b662f3de1b40259e42b0865943ac98`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:20.669703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d8ef398b-bb19-4a15-a2c1-0bc7b3a80cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:20.709174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:20.735703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1095/2178] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:20.746628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:20.748384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:20.805145 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e542-d3c5-4bbe-ac2d-3c5875261434 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:20.809303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:20.809943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:20.810227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:20.810503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:20.865731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:20.867090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:20.876678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1084/2179] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:22.325316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:22.325843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:22.327156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:22.349132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:22.350846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:22.414201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1096/2180] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:22.427807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:22.433197 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:22.496404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:22.512535 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e7ed-8660-4900-bea9-b42648b46722 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:22.518211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:22.518722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:22.519077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:22.519423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:22.531329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:22.532964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:22.541953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1085/2181] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:41:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:41:29.437965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:29.448777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31042c4c-9572-46ed-a88a-0b3bf9d7ceca None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:29.452434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:29.453098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:29.453415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:29.453696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:29.463466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:29.464880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570304b-f193-4781-9c8d-4860c10b13bc service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:29.474945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1097/2182] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:33.054504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=6 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:39.470147 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=41 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:41.676224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:41.737062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1774a7-b2a5-4222-9994-1caff0147366 service swift] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:41.741665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:41.742613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:41.743518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:41.744198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] PATH_INFO: `/v3/users/121c1b61547a482c93113de47152c742` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:41.745736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=121c1b61547a482c93113de47152c742)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:41.747763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:41.917663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '121c1b61547a482c93113de47152c742'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:41.918268 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '121c1b61547a482c93113de47152c742'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:41.919109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '121c1b61547a482c93113de47152c742'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:41.926813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '121c1b61547a482c93113de47152c742'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:41.939876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '121c1b61547a482c93113de47152c742'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.008014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1086/2183] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:41 2022] DELETE /identity/v3/users/121c1b61547a482c93113de47152c742 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:42.012545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:42.012998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:42.013107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:42.031120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:42.032829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:42.089613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1098/2184] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:42.099122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:42.100524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:42.161687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ce414-83a2-4f80-97d7-7b983ae01a1f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:42.165320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:42.165964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:42.166246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:42.166601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:42.176170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:42.177580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:42.187157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1087/2185] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:42.589933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:42.592202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:42.606464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318c01f-3624-4520-af76-39983be46fa8 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:42.611749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:42.612766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:42.613191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:42.613795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] PATH_INFO: `/v3/projects/e898ce340a7a4ac6af1639a02ebe9ef1` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:42.615135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e898ce340a7a4ac6af1639a02ebe9ef1)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:42.617285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:42.747597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e898ce340a7a4ac6af1639a02ebe9ef1', 'request_id': 'req-691902f4-259e-4f9d-ab68-87cc919b7d3c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.754227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e898ce340a7a4ac6af1639a02ebe9ef1', 'request_id': 'req-691902f4-259e-4f9d-ab68-87cc919b7d3c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.852754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e898ce340a7a4ac6af1639a02ebe9ef1', 'request_id': 'req-691902f4-259e-4f9d-ab68-87cc919b7d3c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.864786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] The token cache is being invalidate because project e898ce340a7a4ac6af1639a02ebe9ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:42.865078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.865749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:42.867299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1099/2186] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:42 2022] DELETE /identity/v3/projects/e898ce340a7a4ac6af1639a02ebe9ef1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:42.872123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:42.884148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b5df-a0c3-47a3-9cb1-21e8f0e99a61 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:42.924592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:42.985827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:42.987399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:42.987755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:42.988248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] PATH_INFO: `/v3/users/f4b662f3de1b40259e42b0865943ac98` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:42.989724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b662f3de1b40259e42b0865943ac98)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:42.996019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:43.202618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b662f3de1b40259e42b0865943ac98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:43.203390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b662f3de1b40259e42b0865943ac98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:43.203996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b662f3de1b40259e42b0865943ac98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:43.212042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b662f3de1b40259e42b0865943ac98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:43.224514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-37c8f2bb-8181-4ecc-b45f-e1b36a1adca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b662f3de1b40259e42b0865943ac98'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:43.321885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1088/2187] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:42 2022] DELETE /identity/v3/users/f4b662f3de1b40259e42b0865943ac98 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:43.789829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:43.791270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:43.807090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691902f4-259e-4f9d-ab68-87cc919b7d3c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:43.864498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:43.865383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:43.865670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:43.865948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] PATH_INFO: `/v3/projects/b0851abc260146e0861207d4757e7fcf` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:43.867139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0851abc260146e0861207d4757e7fcf)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:43.869120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:43.969070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:43.994738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0851abc260146e0861207d4757e7fcf', 'request_id': 'req-f576ea63-9965-4f32-9fb1-abbe9078dbe5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:44.004033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0851abc260146e0861207d4757e7fcf', 'request_id': 'req-f576ea63-9965-4f32-9fb1-abbe9078dbe5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:44.047059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0851abc260146e0861207d4757e7fcf', 'request_id': 'req-f576ea63-9965-4f32-9fb1-abbe9078dbe5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:44.061724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] The token cache is being invalidate because project b0851abc260146e0861207d4757e7fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:44.062104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:44.062917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f576ea63-9965-4f32-9fb1-abbe9078dbe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:44.064770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1100/2188] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:43 2022] DELETE /identity/v3/projects/b0851abc260146e0861207d4757e7fcf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:44.070793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.071919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.072266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.090643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:44.093039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:44.166328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1089/2189] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:44.176152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.176554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.176905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.195690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:44.197507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:44.205508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1101/2190] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:44.209622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.219482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab11f7-9702-40d6-974b-8bbd6e65524d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.224349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.225072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.225354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.225639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.226371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:44.226680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.228326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.240395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1090/2191] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:41:44.248565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.258149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2ebf-3b91-47fa-844f-1b68fa055c15 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.263560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.264218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.264499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.264775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.265506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.267477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.339501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1102/2192] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:44 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:44.345993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.355991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517976-8f62-49de-8de9-41ae96a851e5 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.382357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:44.412824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.413518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.413805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.414082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.415324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.417336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.505616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1091/2193] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:44 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:44.509703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.520588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fbe13-1936-4963-8324-1e723cd221d6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.525292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.525956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.526245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.526532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.527262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.528855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.540145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1103/2194] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:44.543965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.554449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266be2f1-f667-4c60-ae68-ad527b7ec7c9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.560096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.560858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.561326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.561609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.562343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.563934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee3bb-1911-4350-9ad9-744f379d42ce admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.577354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1092/2195] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:44.578671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.593839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ff39a-d9cb-4d66-97f8-283c7695bd64 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.598930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.603792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.603792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.603792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] PATH_INFO: `/v3/projects/22e9e3a22c9146a586e2daaa7a5b7b0e/users/9546cf5a9e124290bdd9f312047bb6c0/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.615865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e9e3a22c9146a586e2daaa7a5b7b0e, user_id=9546cf5a9e124290bdd9f312047bb6c0, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.619263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.738454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1104/2196] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:44 2022] PUT /identity/v3/projects/22e9e3a22c9146a586e2daaa7a5b7b0e/users/9546cf5a9e124290bdd9f312047bb6c0/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:44.750612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.750906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.751191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.784407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] MFA Rules not processed for user `9546cf5a9e124290bdd9f312047bb6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:44.785668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:44.853507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1093/2197] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:44.864769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:44.866820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:44.928490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3a55-4967-4b39-868e-2be160260e7a admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:44.932981 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:44.933749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:44.934033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:44.934313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:44.943179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:44.944689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fef6dd-6e2a-41ba-9989-acd0a9e7f6b5 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:44.955525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1105/2198] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:45.847155 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:45.850022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:45.913520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25085cb7-cf24-4e1a-879f-013409958003 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:45.918692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:45.919626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:45.919925 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:45.920195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] PATH_INFO: `/v3/users/a26b10a93dc7436bad2ce5a8e9b16d04` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:45.921448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26b10a93dc7436bad2ce5a8e9b16d04)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:45.923604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:46.083322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:46.089582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26b10a93dc7436bad2ce5a8e9b16d04'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:46.090238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26b10a93dc7436bad2ce5a8e9b16d04'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:46.090931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26b10a93dc7436bad2ce5a8e9b16d04'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:46.097884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26b10a93dc7436bad2ce5a8e9b16d04'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:46.110178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26b10a93dc7436bad2ce5a8e9b16d04'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:46.195322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1094/2199] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:45 2022] DELETE /identity/v3/users/a26b10a93dc7436bad2ce5a8e9b16d04 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:46.200259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:46.200792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:46.200960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:46.219352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:46.220595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:46.279519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:46.294336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1106/2200] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:46.303982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:46.306440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:46.363434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ea466-034e-4104-b458-24e2e5b39b10 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:46.367152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:46.367841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:46.368118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:46.368438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:46.377696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:46.379105 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:46.388355 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1095/2201] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:46.917513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:46.918943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:46.935952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190db365-33ef-448f-b25c-824e900eeb57 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:46.942365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:46.943237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:46.943522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:46.943804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] PATH_INFO: `/v3/projects/de5fc0bb321b4d02a13fadc91a7b7e52` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:46.945042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5fc0bb321b4d02a13fadc91a7b7e52)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:46.947578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:47.060385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5fc0bb321b4d02a13fadc91a7b7e52', 'request_id': 'req-f6326248-ba6b-497a-a7e2-68742d0c366f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.068815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5fc0bb321b4d02a13fadc91a7b7e52', 'request_id': 'req-f6326248-ba6b-497a-a7e2-68742d0c366f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.121149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5fc0bb321b4d02a13fadc91a7b7e52', 'request_id': 'req-f6326248-ba6b-497a-a7e2-68742d0c366f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.134465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] The token cache is being invalidate because project de5fc0bb321b4d02a13fadc91a7b7e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:47.134745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.135392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.136986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1107/2202] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:46 2022] DELETE /identity/v3/projects/de5fc0bb321b4d02a13fadc91a7b7e52 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:47.140916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:47.150846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ac366-4240-4892-961b-11a9c57c37d0 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:47.210311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:47.211221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:47.211527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:47.211804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] PATH_INFO: `/v3/users/7646afb7b6f7474698b3dad8787636f9` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:47.213008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7646afb7b6f7474698b3dad8787636f9)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:47.214962 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:47.430823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7646afb7b6f7474698b3dad8787636f9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.431488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7646afb7b6f7474698b3dad8787636f9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.432094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7646afb7b6f7474698b3dad8787636f9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.441643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7646afb7b6f7474698b3dad8787636f9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.446732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:47.455169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b24419e-1fda-4355-b047-422ef4f8b275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7646afb7b6f7474698b3dad8787636f9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:47.543072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1096/2203] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:41:47 2022] DELETE /identity/v3/users/7646afb7b6f7474698b3dad8787636f9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:48.090900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.090900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:48.103059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6326248-ba6b-497a-a7e2-68742d0c366f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.162536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.163324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.163608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.163881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] PATH_INFO: `/v3/projects/d03e2d8fdde446e9a780a46a4f353403` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.165742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03e2d8fdde446e9a780a46a4f353403)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.167657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.275248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03e2d8fdde446e9a780a46a4f353403', 'request_id': 'req-a61984bc-e919-4315-b478-c2d9128623cd'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:48.281964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03e2d8fdde446e9a780a46a4f353403', 'request_id': 'req-a61984bc-e919-4315-b478-c2d9128623cd'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:48.337678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03e2d8fdde446e9a780a46a4f353403', 'request_id': 'req-a61984bc-e919-4315-b478-c2d9128623cd'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:48.349510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] The token cache is being invalidate because project d03e2d8fdde446e9a780a46a4f353403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:41:48.349798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:48.350437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a61984bc-e919-4315-b478-c2d9128623cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:41:48.351969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1108/2204] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:41:48 2022] DELETE /identity/v3/projects/d03e2d8fdde446e9a780a46a4f353403 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:48.357365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.357867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.357978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.376414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:48.377680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:48.449820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1097/2205] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:48.457494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.461048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.461048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.470564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:48.477962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:48.479367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:48.496709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1109/2206] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:48.496818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.502606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225ad2-f241-4e72-84ef-c18dcbad520c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.507210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.507864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.508150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.508428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.509285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:41:48.509600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.511196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.530921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1098/2207] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:41:48.535156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.545104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808be657-c33a-48c1-bb59-565cae5e56cf None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.550017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.550907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.551214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.551592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.552329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.554698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.663977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1110/2208] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:48 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:48.668687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.684266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549c094-282a-4bb5-a85d-72bd7a092c90 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.738902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.741494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.741494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.741494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.741494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.742791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.818189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1099/2209] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:48 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:48.822486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.833254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817031c-2cfb-4225-936d-8c913ead582c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.838464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.839113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.839554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.840192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.841003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.843593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.860260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1111/2210] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:48.864162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.874706 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbea67-8036-4a74-b67f-e33e452a255c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.880101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.881060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.881060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.881337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.882168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.884306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e630cd-43b2-4092-8fd9-7c351f93be4a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:48.894984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1100/2211] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:48.900305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:48.910816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b500-2ced-460b-a9cd-f61f06662f2b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:48.915568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:48.916451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:48.916737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:48.917016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] PATH_INFO: `/v3/projects/f59dd722a1cc4aa0b5a124aae01289ce/users/6c5ff19e165c41179c99d13fe6d25db5/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:48.932185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59dd722a1cc4aa0b5a124aae01289ce, user_id=6c5ff19e165c41179c99d13fe6d25db5, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:48.935064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.036268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1112/2212] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:48 2022] PUT /identity/v3/projects/f59dd722a1cc4aa0b5a124aae01289ce/users/6c5ff19e165c41179c99d13fe6d25db5/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:49.046855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.047149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.047427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.080356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] MFA Rules not processed for user `6c5ff19e165c41179c99d13fe6d25db5`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:49.081611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:49.114830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:49.148830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1101/2213] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:49.157910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.220753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21252e-3bd1-4465-aab9-7f832a3f59f3 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.227738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.228412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.228703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.228983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.229735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.231666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.342447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1113/2214] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:41:49 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:49.343836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.355066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a6c71-dec5-440f-ae68-9ac9ceddc9a2 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.435421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.436185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.436377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.436653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.442139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.442139 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.529659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1102/2215] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:41:49 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:41:49.533586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.556463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504ff1-4b0e-4f8e-8b2a-c47584956f2c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.566885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.567694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.568045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.568402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.569334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.570986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.583639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1114/2216] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:49.590174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.601398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed36c0-b403-4b75-b744-1904a5e2e6fe admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.608734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.609778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.615339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.615339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] PATH_INFO: `/v3/projects/5c6922c357324c4ab804a330425b61de/users/f584d7a19e3f4946b45cb46dc0f3ce5d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.628100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6922c357324c4ab804a330425b61de, user_id=f584d7a19e3f4946b45cb46dc0f3ce5d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.630918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.739030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1103/2217] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:49 2022] PUT /identity/v3/projects/5c6922c357324c4ab804a330425b61de/users/f584d7a19e3f4946b45cb46dc0f3ce5d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:41:49.743070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.754588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c480-e4c3-454d-af9f-fa1ea4d6ae9d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.811917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.812587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.812870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.813146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.813902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.815649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577e3e6-9d23-48ac-93fa-b523cfdfa415 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.828029 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1115/2218] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:41:49.831850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.843850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287aebd-179a-438c-9637-c926355bc30a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:49.848870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:49.850196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.850325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.850545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] PATH_INFO: `/v3/projects/5c6922c357324c4ab804a330425b61de/users/f584d7a19e3f4946b45cb46dc0f3ce5d/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.852759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6922c357324c4ab804a330425b61de, user_id=f584d7a19e3f4946b45cb46dc0f3ce5d, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:49.855783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:49.868614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1104/2219] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:41:49 2022] PUT /identity/v3/projects/5c6922c357324c4ab804a330425b61de/users/f584d7a19e3f4946b45cb46dc0f3ce5d/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:41:49.880293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:49.880961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:49.881410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:49.911692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:49.914890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] MFA Rules not processed for user `f584d7a19e3f4946b45cb46dc0f3ce5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:49.916206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:49.984149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1116/2220] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:41:49.995916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:49.997634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:50.059114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b299e3-be8a-4043-a698-b0d923f7c33e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:50.062840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:50.065581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:50.065581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:50.065581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:50.074950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:50.075973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb8bf-dbb6-45ba-b0a1-c055ee3547db service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:50.087750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1105/2221] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:50.467476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:50.469858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:50.483765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72452075-b011-4e64-9cfe-7e838612f853 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:50.487579 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:50.488233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:50.488517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:50.488798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:50.551764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:50.553380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:50.563977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1117/2222] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:52.219078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141f31-4faa-4c22-a530-099ae67afbe1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:52.219570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141f31-4faa-4c22-a530-099ae67afbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:52.219666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141f31-4faa-4c22-a530-099ae67afbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:52.237811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-68141f31-4faa-4c22-a530-099ae67afbe1 None None] MFA Rules not processed for user `97f1b1e52a984639a504441e3701b981`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:52.239016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-68141f31-4faa-4c22-a530-099ae67afbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:52.263187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:52.297944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1106/2223] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:41:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:00.559063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=56 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:00.595964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:00.703988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ae075-53b5-4d06-adbc-9d7af6fd377c service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:00.708966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:00.710211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:00.710688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:00.710966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] PATH_INFO: `/v3/users/6c5ff19e165c41179c99d13fe6d25db5` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:00.712768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5ff19e165c41179c99d13fe6d25db5)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:00.714766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:00.924426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5ff19e165c41179c99d13fe6d25db5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:00.925077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5ff19e165c41179c99d13fe6d25db5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:00.925702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5ff19e165c41179c99d13fe6d25db5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:00.932584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5ff19e165c41179c99d13fe6d25db5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:00.945535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5ff19e165c41179c99d13fe6d25db5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:01.023221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1118/2224] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:00 2022] DELETE /identity/v3/users/6c5ff19e165c41179c99d13fe6d25db5 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:01.028414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:01.028897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:01.028998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:01.056509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:01.058110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:01.124426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1107/2225] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:01.134430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:01.135881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:01.194196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6deb-689c-4acf-bcdd-f55555778e2f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:01.198433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:01.199135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:01.199528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:01.199796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:01.212308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:01.213790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:01.223743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1119/2226] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:01.667063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:01.676902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2465cb0-4dc7-473c-8edb-d7616a63c54f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:01.681400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:01.682057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:01.682363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:01.682618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:01.684509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:01.737630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:01.739841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:01.760977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:01.762595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:01.779283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1108/2227] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:01.797712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda60e4-ff87-418a-9ed6-59cd051cf38d service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:01.799806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:01.800672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:01.800959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:01.801275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] PATH_INFO: `/v3/projects/f59dd722a1cc4aa0b5a124aae01289ce` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:01.802668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59dd722a1cc4aa0b5a124aae01289ce)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:01.804931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:01.825420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:01.939368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59dd722a1cc4aa0b5a124aae01289ce', 'request_id': 'req-36a105eb-4d22-4420-84b7-18732d6b2053'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:01.945764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59dd722a1cc4aa0b5a124aae01289ce', 'request_id': 'req-36a105eb-4d22-4420-84b7-18732d6b2053'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.028714 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59dd722a1cc4aa0b5a124aae01289ce', 'request_id': 'req-36a105eb-4d22-4420-84b7-18732d6b2053'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.041057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] The token cache is being invalidate because project f59dd722a1cc4aa0b5a124aae01289ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:02.041376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.042047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.043673 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1120/2228] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:01 2022] DELETE /identity/v3/projects/f59dd722a1cc4aa0b5a124aae01289ce => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:02.047734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:02.058371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf4342-f451-456f-8c59-ed7fdc1aef74 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:02.116143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:02.117069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:02.117349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:02.117812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] PATH_INFO: `/v3/users/f584d7a19e3f4946b45cb46dc0f3ce5d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:02.118879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f584d7a19e3f4946b45cb46dc0f3ce5d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:02.121086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:02.132577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:02.350128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f584d7a19e3f4946b45cb46dc0f3ce5d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.350883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f584d7a19e3f4946b45cb46dc0f3ce5d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.351553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f584d7a19e3f4946b45cb46dc0f3ce5d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.359956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f584d7a19e3f4946b45cb46dc0f3ce5d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.380572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-e3f217db-75d6-4395-81fe-9740a485893c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f584d7a19e3f4946b45cb46dc0f3ce5d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:02.460030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1109/2229] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:02 2022] DELETE /identity/v3/users/f584d7a19e3f4946b45cb46dc0f3ce5d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:02.997750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:03.000169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:03.035986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a105eb-4d22-4420-84b7-18732d6b2053 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:03.099469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:03.100337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.100621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.100899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] PATH_INFO: `/v3/projects/5c6922c357324c4ab804a330425b61de` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.102132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6922c357324c4ab804a330425b61de)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:03.104586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:03.261808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6922c357324c4ab804a330425b61de', 'request_id': 'req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:03.267756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6922c357324c4ab804a330425b61de', 'request_id': 'req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:03.354843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6922c357324c4ab804a330425b61de', 'request_id': 'req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:03.372784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] The token cache is being invalidate because project 5c6922c357324c4ab804a330425b61de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:03.373224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:03.374096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0b1ebb37-08d7-4922-b2db-f4a156e0e54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:03.376816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1121/2230] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:02 2022] DELETE /identity/v3/projects/5c6922c357324c4ab804a330425b61de => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:03.381418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.382654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.382654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.413478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:03.415631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:03.539678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1110/2231] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:03.544460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.545287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.546122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.569698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:03.570970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:03.579489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1122/2232] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:03.583612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:03.597620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440e686-2930-4cf0-be0f-c7b4d3f20b2c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:03.605566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:03.606364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.606747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.607201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.608547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:42:03.608547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:03.610562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:03.624542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1111/2233] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:42:03.628915 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:03.640306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39484-000e-4532-9901-ba8b3663e144 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:03.646345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:03.647421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.647903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.648353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.649869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:03.652322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:03.763039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1123/2234] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:03 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:03.766781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:03.777533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa4a5-9cb7-42ef-a52d-76c49fb4ba2c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:03.843835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:03.853579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.853579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:03.853579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:03.853579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:03.853579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:03.960982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1112/2235] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:03 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:03.965357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:03.985704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d4021-ddc8-45f9-a06a-eae1d4f624cc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:03.998846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:03.999542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:03.999831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.000277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.004327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.006046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.007578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:04.024870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1124/2236] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:04.026385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.034226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:04.037900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b4ad5-d4c0-4931-bd8f-fbb3070547de admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.043980 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.044695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.044984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.045302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.046045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.047645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911fdc-0cca-4573-81e6-d1047f4a7d77 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.059131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1113/2237] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:04.063384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.076441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb48f-c0a9-440c-b39f-5b52dafb107b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.081666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.083527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.083871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.084489 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] PATH_INFO: `/v3/projects/c1f4bc341a104277bc2518a1ef5b6a6d/users/491cfca94e6a49f7a8eb491e8ed39dfb/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.103551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f4bc341a104277bc2518a1ef5b6a6d, user_id=491cfca94e6a49f7a8eb491e8ed39dfb, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.106408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.193610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1125/2238] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:04 2022] PUT /identity/v3/projects/c1f4bc341a104277bc2518a1ef5b6a6d/users/491cfca94e6a49f7a8eb491e8ed39dfb/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:04.195588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.196582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.196931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.228334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] MFA Rules not processed for user `491cfca94e6a49f7a8eb491e8ed39dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:04.229684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:04.311926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1114/2239] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:04.315710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.402571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a661b4c-7c82-4fd3-8e88-91be0d5ef19f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.407652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.408500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.408877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.409288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.410102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.412144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.550824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1126/2240] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:04 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:04.557973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.565466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24b0f0-4a28-400c-86cb-9f8cf27a8198 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.741936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.742626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.743539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.743781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.744735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.748804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.864517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1115/2241] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:04 2022] POST /identity/v3/users => generated 299 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:04.867507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.887132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc3e6a-9a3d-484d-9f3b-d6e61281f06d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.892836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.893642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.894018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.894409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.895353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.897292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:04.925426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1127/2242] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:42:04.926678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:04.938108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f18ce-f334-4ccf-aec7-5c17daa9a8d9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:04.943580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:04.944837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:04.949826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:04.949826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] PATH_INFO: `/v3/projects/a0256020ffe64727a1f60d84f683dba8/users/286d619e3d0c4bcbbf11d0f662033526/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:04.984964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0256020ffe64727a1f60d84f683dba8, user_id=286d619e3d0c4bcbbf11d0f662033526, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:04.987958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:05.059234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1116/2243] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:04 2022] PUT /identity/v3/projects/a0256020ffe64727a1f60d84f683dba8/users/286d619e3d0c4bcbbf11d0f662033526/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:05.062995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:05.075871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874ccf8-21ed-4180-aaf8-a7ce6c6fd5b5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:05.143434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:05.144803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:05.145299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:05.151877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:05.151877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:05.151877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb6518-8e87-40c8-8079-3760b2cc9e42 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:05.160351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1128/2244] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:05.169798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:05.180434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da97b-feb2-4ddf-bd7a-d610a856c8da admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:05.185123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:05.186134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:05.186397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:05.186679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] PATH_INFO: `/v3/projects/a0256020ffe64727a1f60d84f683dba8/users/286d619e3d0c4bcbbf11d0f662033526/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:05.188703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0256020ffe64727a1f60d84f683dba8, user_id=286d619e3d0c4bcbbf11d0f662033526, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:05.191391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:05.202759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1117/2245] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:05 2022] PUT /identity/v3/projects/a0256020ffe64727a1f60d84f683dba8/users/286d619e3d0c4bcbbf11d0f662033526/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:05.214736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:05.214736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:05.214736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:05.263672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] MFA Rules not processed for user `286d619e3d0c4bcbbf11d0f662033526`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:05.265725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:05.377460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1129/2246] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:05.393480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:05.395652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:05.466565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95191312-2f06-41fd-adf6-1629ec6f6fc9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:05.470322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:05.473226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:05.473226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:05.473226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:05.480902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:05.482380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d1e33-9efd-45f8-9bd3-79f857ac1648 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:05.492350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1118/2247] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:06.388563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:06.474724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:06.906689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:06.973628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a29d8-3e32-4c78-82a4-91a48388a355 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:06.978789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:06.979664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:06.979949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:06.980229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] PATH_INFO: `/v3/users/491cfca94e6a49f7a8eb491e8ed39dfb` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:06.981602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=491cfca94e6a49f7a8eb491e8ed39dfb)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:06.983535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:07.187317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491cfca94e6a49f7a8eb491e8ed39dfb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:07.187989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491cfca94e6a49f7a8eb491e8ed39dfb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:07.188593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491cfca94e6a49f7a8eb491e8ed39dfb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:07.195713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491cfca94e6a49f7a8eb491e8ed39dfb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:07.210958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491cfca94e6a49f7a8eb491e8ed39dfb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:07.281315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1130/2248] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:06 2022] DELETE /identity/v3/users/491cfca94e6a49f7a8eb491e8ed39dfb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:07.286089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:07.286629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:07.287210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:07.310077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:07.311500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:07.372677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1119/2249] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:07.383671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:07.385093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:07.458769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce029-9449-4bb0-9a71-de4cafaba5fc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:07.463366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:07.463972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:07.464258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:07.464594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:07.474631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:07.476090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:07.487084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1131/2250] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:07.589272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:07.950796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:07.952906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:07.967734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4d467-1355-4eae-8583-65e969af707e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:07.972873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:07.973825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:07.974451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:07.974451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] PATH_INFO: `/v3/projects/c1f4bc341a104277bc2518a1ef5b6a6d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:07.984398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f4bc341a104277bc2518a1ef5b6a6d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:07.984398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:08.129066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f4bc341a104277bc2518a1ef5b6a6d', 'request_id': 'req-881ef0af-8cec-411c-ba44-b44d3c31be95'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.141507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f4bc341a104277bc2518a1ef5b6a6d', 'request_id': 'req-881ef0af-8cec-411c-ba44-b44d3c31be95'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.204782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f4bc341a104277bc2518a1ef5b6a6d', 'request_id': 'req-881ef0af-8cec-411c-ba44-b44d3c31be95'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.225026 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] The token cache is being invalidate because project c1f4bc341a104277bc2518a1ef5b6a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:08.226359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.232500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.235008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1120/2251] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:07 2022] DELETE /identity/v3/projects/c1f4bc341a104277bc2518a1ef5b6a6d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:08.241032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:08.257660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13336cc2-65b1-4441-bfc5-2cdd1e0e47b5 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:08.345646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:08.346300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:08.346581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:08.346943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] PATH_INFO: `/v3/users/286d619e3d0c4bcbbf11d0f662033526` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:08.348184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286d619e3d0c4bcbbf11d0f662033526)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:08.350628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:08.542501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286d619e3d0c4bcbbf11d0f662033526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.544423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286d619e3d0c4bcbbf11d0f662033526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.545144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286d619e3d0c4bcbbf11d0f662033526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.556841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286d619e3d0c4bcbbf11d0f662033526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.574319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c1a81792-e9b4-4281-90c7-5666c1e239b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286d619e3d0c4bcbbf11d0f662033526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:08.638478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1132/2252] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:08 2022] DELETE /identity/v3/users/286d619e3d0c4bcbbf11d0f662033526 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:09.270127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:09.271617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:09.285930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ef0af-8cec-411c-ba44-b44d3c31be95 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:09.342987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:09.343843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.344126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.344396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] PATH_INFO: `/v3/projects/a0256020ffe64727a1f60d84f683dba8` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.345600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0256020ffe64727a1f60d84f683dba8)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:09.347505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:09.349684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:09.458056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0256020ffe64727a1f60d84f683dba8', 'request_id': 'req-3939476e-2d4f-4141-850d-98d8749f984e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:09.464091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0256020ffe64727a1f60d84f683dba8', 'request_id': 'req-3939476e-2d4f-4141-850d-98d8749f984e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:09.519369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0256020ffe64727a1f60d84f683dba8', 'request_id': 'req-3939476e-2d4f-4141-850d-98d8749f984e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:09.531143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] The token cache is being invalidate because project a0256020ffe64727a1f60d84f683dba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:09.531553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:09.532195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3939476e-2d4f-4141-850d-98d8749f984e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:09.533756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1121/2253] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:09 2022] DELETE /identity/v3/projects/a0256020ffe64727a1f60d84f683dba8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:09.538764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.539055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.539337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.557185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:09.558398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:09.623237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1133/2254] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:09.631469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.631774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.632097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.651267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:09.652487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:09.660775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1122/2255] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:09.665960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:09.675843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef7276-b897-4bec-b2e0-292df53e0fa5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:09.684500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:09.684500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.684500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.684500 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.684882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:42:09.684882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:09.684882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:09.686849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:09.696277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1134/2256] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:42:09.705546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:09.717387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89a21-5fb9-40bf-92ea-9290e73f06e5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:09.724068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:09.725038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.725513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.725941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.727663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:09.730884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:09.819579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1123/2257] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:09 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:09.823940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:09.834730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae123efa-109d-4dc1-97cd-7260d71eb833 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:09.900741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:09.901466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:09.902062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:09.902349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:09.903313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:09.905271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:09.995278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1135/2258] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:09 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:09.999483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:10.012240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02600bbf-72d8-4756-98dd-ec8799530744 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:10.017257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:10.017993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:10.018341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:10.018677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:10.019467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:10.021300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:10.034729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1124/2259] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:10.044026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:10.054277 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb4380-54b8-4579-bbda-be6031856b86 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:10.060177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:10.061505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:10.062274 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:10.062868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:10.064642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:10.067260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69025f48-aa38-4c18-ae3f-927ad07b875f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:10.080186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1136/2260] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:42:10.084260 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:10.095686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200ade-27ee-4a53-bb3c-8c69b5e5ab98 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:10.100516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:10.101486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:10.101790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:10.102451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] PATH_INFO: `/v3/projects/e7d10c6c04bd4ef696f6c971290d665b/users/0d3292c6fb72438ca79d4573760eebb8/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:10.118869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d10c6c04bd4ef696f6c971290d665b, user_id=0d3292c6fb72438ca79d4573760eebb8, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:10.122074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:10.227563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1125/2261] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:10 2022] PUT /identity/v3/projects/e7d10c6c04bd4ef696f6c971290d665b/users/0d3292c6fb72438ca79d4573760eebb8/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:10.238118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845c24b-82ef-4041-b4f9-8fabb4b04fb7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:10.238539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845c24b-82ef-4041-b4f9-8fabb4b04fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:10.238924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845c24b-82ef-4041-b4f9-8fabb4b04fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:10.262888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-6845c24b-82ef-4041-b4f9-8fabb4b04fb7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:10.264323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6845c24b-82ef-4041-b4f9-8fabb4b04fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:10.327160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1137/2262] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:10.336649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:10.338519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:10.398390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0add67b-e71d-4400-ae38-db3afa94524c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:10.402127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:10.402783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:10.403068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:10.403344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:10.412942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:10.414750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:10.425793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1126/2263] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:11.858498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922268-d097-494e-b78d-509a15d7848c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:11.859028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922268-d097-494e-b78d-509a15d7848c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:11.859144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922268-d097-494e-b78d-509a15d7848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:11.896087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-60922268-d097-494e-b78d-509a15d7848c None None] MFA Rules not processed for user `0d3292c6fb72438ca79d4573760eebb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:11.897365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-60922268-d097-494e-b78d-509a15d7848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:11.980176 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1138/2264] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:11.981764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:11.994755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6c7b1-64ec-45f1-9783-af4eb1499569 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.017982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.018683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.018971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.019258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.019979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.022042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.142387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1127/2265] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:11 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:42:12.143705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922268-d097-494e-b78d-509a15d7848c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:12.151737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:12.156265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922268-d097-494e-b78d-509a15d7848c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.238604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.240059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.240059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.240059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.240935 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.243652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.343971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1139/2266] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:12 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:12.348410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:12.363091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169895-820d-42d5-bd43-0b0a1997520f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.370472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.371780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.372271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.372705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.373632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.375705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.379563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:12.389046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1128/2267] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:12.393574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:12.404303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a05a1e-2d9c-4ba9-98eb-c4afff1b7437 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.409037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.409966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.410249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.410555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] PATH_INFO: `/v3/projects/0de42729c04a41b2978b6eb69c2bea61/users/e1ff7e2a0c77497495785dc2dcbafd1c/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.428657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de42729c04a41b2978b6eb69c2bea61, user_id=e1ff7e2a0c77497495785dc2dcbafd1c, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.432313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.497370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1140/2268] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:12 2022] PUT /identity/v3/projects/0de42729c04a41b2978b6eb69c2bea61/users/e1ff7e2a0c77497495785dc2dcbafd1c/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:12.501331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:12.516114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20474-aa80-4c91-9830-97ea48131d49 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.593067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.593837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.594219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.594526 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.595377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.597335 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11eda62-4d73-4f7f-ac9f-063097025a79 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.609924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1129/2269] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:12.614252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:12.625960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36772c8-4aa9-4946-941a-b7f42ada5576 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:12.637131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:12.638097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:12.639457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:12.640051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] PATH_INFO: `/v3/projects/0de42729c04a41b2978b6eb69c2bea61/users/e1ff7e2a0c77497495785dc2dcbafd1c/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:12.642151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de42729c04a41b2978b6eb69c2bea61, user_id=e1ff7e2a0c77497495785dc2dcbafd1c, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:12.644839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:12.657050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1141/2270] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:12 2022] PUT /identity/v3/projects/0de42729c04a41b2978b6eb69c2bea61/users/e1ff7e2a0c77497495785dc2dcbafd1c/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:13.833649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:13.837848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:13.837848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:13.865793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] MFA Rules not processed for user `e1ff7e2a0c77497495785dc2dcbafd1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:13.867014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:13.944385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1130/2271] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:13.955301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:13.957238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:14.016426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ea249-2bfb-4c2c-9e73-73bc96bf45ce admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:14.019980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:14.020661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:14.020941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:14.021256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:14.084448 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:14.087569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:14.098281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1142/2272] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:15.848768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:15.858882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcecea-4a5f-41f0-8a18-122b34d03261 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:15.862439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:15.863076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:15.863475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:15.863757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:15.872969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:15.874420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8b886-a013-49a2-a5a4-6b745caa1723 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:15.884310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1131/2273] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:24.094354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:25.879554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=70 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:31.308050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:31.378588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18e1b0-8044-417a-b357-3ec9451d2ecc service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:31.383379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:31.384359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:31.384614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:31.384889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] PATH_INFO: `/v3/users/0d3292c6fb72438ca79d4573760eebb8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:31.386480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3292c6fb72438ca79d4573760eebb8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:31.388498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:31.619442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3292c6fb72438ca79d4573760eebb8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:31.620374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3292c6fb72438ca79d4573760eebb8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:31.626881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3292c6fb72438ca79d4573760eebb8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:31.640426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3292c6fb72438ca79d4573760eebb8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:31.653951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3292c6fb72438ca79d4573760eebb8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:31.760021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1143/2274] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:31 2022] DELETE /identity/v3/users/0d3292c6fb72438ca79d4573760eebb8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:31.877286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:31.877286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:31.877286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:31.894400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:31.895724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:31.955003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1132/2275] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:31.995570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:31.997098 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:32.007097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:32.010077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:32.070097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddd204-eed1-40bd-9ec6-ddea404afedb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:32.075269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:32.075920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:32.076204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:32.076483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:32.084882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:32.086481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:32.093588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf84e7-41bc-4d38-abea-4e1cdffcfb4b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:32.095145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1144/2276] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:42:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:42:32.098957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:32.099946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:32.100413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:32.100761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] PATH_INFO: `/v3/projects/e7d10c6c04bd4ef696f6c971290d665b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:32.102080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d10c6c04bd4ef696f6c971290d665b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:32.104558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:32.221813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d10c6c04bd4ef696f6c971290d665b', 'request_id': 'req-bb11ea57-d034-4fc7-949e-86ed665620da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.228656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d10c6c04bd4ef696f6c971290d665b', 'request_id': 'req-bb11ea57-d034-4fc7-949e-86ed665620da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.299548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d10c6c04bd4ef696f6c971290d665b', 'request_id': 'req-bb11ea57-d034-4fc7-949e-86ed665620da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.312701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] The token cache is being invalidate because project e7d10c6c04bd4ef696f6c971290d665b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:32.313130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.313906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.315528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1133/2277] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:32 2022] DELETE /identity/v3/projects/e7d10c6c04bd4ef696f6c971290d665b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:32.319678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:32.331469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9d48-e37c-4a6f-9956-2453e843599c service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:32.337017 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:32.410241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:32.411145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:32.411429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:32.411701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] PATH_INFO: `/v3/users/e1ff7e2a0c77497495785dc2dcbafd1c` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:32.412931 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ff7e2a0c77497495785dc2dcbafd1c)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:32.415118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:32.890720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ff7e2a0c77497495785dc2dcbafd1c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.893675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ff7e2a0c77497495785dc2dcbafd1c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.893675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ff7e2a0c77497495785dc2dcbafd1c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.912821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ff7e2a0c77497495785dc2dcbafd1c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:32.939133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eed8d42e-535a-41ff-8981-c322e0e50f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ff7e2a0c77497495785dc2dcbafd1c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.046676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1145/2278] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:32 2022] DELETE /identity/v3/users/e1ff7e2a0c77497495785dc2dcbafd1c => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:33.361732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:33.364368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:33.399274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ea57-d034-4fc7-949e-86ed665620da admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:33.453869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:33.488895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:33.490568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:33.490963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:33.491411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] PATH_INFO: `/v3/projects/0de42729c04a41b2978b6eb69c2bea61` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:33.494441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de42729c04a41b2978b6eb69c2bea61)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:33.496675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:33.627230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de42729c04a41b2978b6eb69c2bea61', 'request_id': 'req-4615a39c-3d28-4b0e-a448-88a3839a4f31'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.633840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de42729c04a41b2978b6eb69c2bea61', 'request_id': 'req-4615a39c-3d28-4b0e-a448-88a3839a4f31'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.696900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de42729c04a41b2978b6eb69c2bea61', 'request_id': 'req-4615a39c-3d28-4b0e-a448-88a3839a4f31'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.711802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] The token cache is being invalidate because project 0de42729c04a41b2978b6eb69c2bea61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:33.712241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.712945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4615a39c-3d28-4b0e-a448-88a3839a4f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:33.714798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1134/2279] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:33 2022] DELETE /identity/v3/projects/0de42729c04a41b2978b6eb69c2bea61 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:33.719783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:33.720395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:33.720395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:33.750235 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:33.752030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:33.826567 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1146/2280] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:33.835611 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:33.836224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:33.836586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:33.859101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:33.860330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:33.873582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1135/2281] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:33.874927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:33.885662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa60d2-955a-46bf-ab09-c61e6f6224b4 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:33.892536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:33.896578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:33.896578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:33.896578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:33.896578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:42:33.896578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:33.897015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:33.916778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1147/2282] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:42:33.919907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:33.933303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2143cd-bb14-4859-ba5d-e64d143c264b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:33.938182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:33.938883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:33.939246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:33.939501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:33.940286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:33.942333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.040070 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1136/2283] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:33 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:42:34.042998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:34.059118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf24ce-9176-432f-8467-033a8361f307 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:34.077455 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:34.179858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:34.180602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.180893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.181227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.181939 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:34.184043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.288232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1148/2284] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:34 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:34.289456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:34.313061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6194e9-6152-46d6-91f7-1d91f4fe3bd0 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:34.317997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:34.318726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.319013 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.319291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.320247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:34.324210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.336339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1137/2285] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:42:34.340961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:34.354930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf01c49-bc5c-45f4-ab52-bb8bd57266e5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:34.361477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:34.362282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.362472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.362755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.363539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:34.365188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6d3bd-b106-421f-b807-ba2761b5a78d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.377215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1149/2286] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:34.381389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:34.392610 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ccd19-859d-4dbc-94fc-a9f5061d733f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:34.398003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:34.399436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.400772 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.401062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] PATH_INFO: `/v3/projects/1f5afd9a249541c98259fc9da5431971/users/5a85e188ea424fa2b3d5f725c0801f6b/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.420410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5afd9a249541c98259fc9da5431971, user_id=5a85e188ea424fa2b3d5f725c0801f6b, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:34.423538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.486880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1138/2287] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:34 2022] PUT /identity/v3/projects/1f5afd9a249541c98259fc9da5431971/users/5a85e188ea424fa2b3d5f725c0801f6b/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:34.503472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.504094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.504663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.546337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] MFA Rules not processed for user `5a85e188ea424fa2b3d5f725c0801f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:34.547615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:34.637691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1150/2288] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:34.649110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:34.651924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:34.725216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af1b2-8065-4eee-9f37-aa9ffb9e9908 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:34.729530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:34.730325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:34.730725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:34.731101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:34.742787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:34.744628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01788764-9d60-4227-8010-13c9675cb8b0 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:34.757988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1139/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:36.896913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:36.898491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:37.087531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c52c0-3fee-45f5-9873-04a7d24b4125 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:37.105802 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:37.107324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:37.107788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:37.108813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] PATH_INFO: `/v3/users/9546cf5a9e124290bdd9f312047bb6c0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:37.110918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9546cf5a9e124290bdd9f312047bb6c0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:37.115985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:37.223244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:37.333363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9546cf5a9e124290bdd9f312047bb6c0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:37.334200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9546cf5a9e124290bdd9f312047bb6c0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:37.334843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9546cf5a9e124290bdd9f312047bb6c0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:37.347445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9546cf5a9e124290bdd9f312047bb6c0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:37.368272 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9546cf5a9e124290bdd9f312047bb6c0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:37.460321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1151/2290] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:36 2022] DELETE /identity/v3/users/9546cf5a9e124290bdd9f312047bb6c0 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:37.461635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:37.462140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:37.462782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:37.472357 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:37.515218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:37.516474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:37.607786 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1140/2291] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:42:37.618092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:37.620049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:37.692471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52768086-a251-49d8-b227-a3da7f8e1b31 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:37.697377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:37.698126 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:37.698487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:37.698838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:37.713713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:37.716144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fead0-8e4c-4f77-9728-74d3ad9633f7 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:37.730527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1152/2292] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:38.292712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:38.299650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:38.344383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e6763-1791-4e36-b3d1-4ecec3936eb8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:38.355620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:38.358494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:38.359196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:38.359617 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] PATH_INFO: `/v3/projects/22e9e3a22c9146a586e2daaa7a5b7b0e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:38.361133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e9e3a22c9146a586e2daaa7a5b7b0e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:38.363436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:38.512243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e9e3a22c9146a586e2daaa7a5b7b0e', 'request_id': 'req-d8d21a79-4496-4c09-9739-72b69dae37d9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:38.532202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e9e3a22c9146a586e2daaa7a5b7b0e', 'request_id': 'req-d8d21a79-4496-4c09-9739-72b69dae37d9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:38.947674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e9e3a22c9146a586e2daaa7a5b7b0e', 'request_id': 'req-d8d21a79-4496-4c09-9739-72b69dae37d9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:38.959385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] The token cache is being invalidate because project 22e9e3a22c9146a586e2daaa7a5b7b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:38.959673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:38.960300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d8d21a79-4496-4c09-9739-72b69dae37d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:38.961975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1141/2293] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:38 2022] DELETE /identity/v3/projects/22e9e3a22c9146a586e2daaa7a5b7b0e => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:38.966927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:38.967473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:38.967614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:38.985478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:38.986630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:39.073962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1153/2294] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:39.082423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.082802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.083149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.107056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:39.108346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:39.122733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1142/2295] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:42:39.123976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.135612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96208-45fb-433e-9fad-ab743c528114 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.141689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.142367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.142648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.143052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.143821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:42:39.144144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.145817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.167060 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1154/2296] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:42:39.168854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.179690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d76593-4be9-487f-b5ca-e6d2647593dc None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.185095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.185989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.186336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.189189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.189189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.189747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.271737 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1143/2297] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:39 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:42:39.275621 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.288127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ef32b-c381-43d3-a1dc-cce8176a5bef admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.333580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:39.373294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.374095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.374438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.374736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.375429 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.377446 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.475962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1155/2298] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:39 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:39.480381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.492162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96d16e-6b3d-4af1-9d73-937aa4b1e05a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.497253 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.497929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.498237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.498497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.499405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.501010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.513468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1144/2299] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:39.517490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.529643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c373f-0788-419a-aca3-11d913349451 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.534685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.535358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.535646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.535927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.536661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.539490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d204c-d5ff-4ee2-9588-698f281b246c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.553191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1156/2300] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:42:39.556235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.568200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2cd9-6f94-4532-8be1-3e7f30545129 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.573887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.575061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.575785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.576164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] PATH_INFO: `/v3/projects/220721ec418148c0b8efff4038b3e1eb/users/6ff37b01a3604cfbb9e98b0003e403eb/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.595229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220721ec418148c0b8efff4038b3e1eb, user_id=6ff37b01a3604cfbb9e98b0003e403eb, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.599130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.706076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1145/2301] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:42:39 2022] PUT /identity/v3/projects/220721ec418148c0b8efff4038b3e1eb/users/6ff37b01a3604cfbb9e98b0003e403eb/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:39.708004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.708320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.708587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.743471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] MFA Rules not processed for user `6ff37b01a3604cfbb9e98b0003e403eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:39.744914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:39.835290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1157/2302] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:39.849321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:39.851276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:39.882999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:39.940476 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81395ce6-3f6a-4428-a0ce-6427439824f1 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:39.945348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:39.946049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:39.946376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:39.947193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:39.959054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:39.961687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752e431-da32-4d42-a952-4424e8c920d6 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:39.982386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1146/2303] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:49.831676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:49.996416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=15 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:50.244423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:50.246420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:50.316560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230e004-327a-4c73-95d1-87ff8b2cae55 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:50.321488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:50.322131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:50.322411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:50.322684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:50.324454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:50.336008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:50.337650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caf243-92f6-4852-a8a7-ea086ca030bd service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:50.348937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1158/2304] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:52.990380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:52.995797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:52.995797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:52.995797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1147/2305] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:42:52.998472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900ba0-69e0-466f-9f68-78f02dba43ab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:52.998951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900ba0-69e0-466f-9f68-78f02dba43ab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:52.999053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba900ba0-69e0-466f-9f68-78f02dba43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.017127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ba900ba0-69e0-466f-9f68-78f02dba43ab None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:53.018403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ba900ba0-69e0-466f-9f68-78f02dba43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:53.090374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1159/2306] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:42:53.106749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:53.109580 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:53.166127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911560f1-1c03-43f4-9452-bdc2de7a6693 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:53.170337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:53.170970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.171250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.171528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.181815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:53.183295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6114db0-7c0e-4659-9e35-87ce4c32b84d service glance] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:53.194121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1148/2307] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:42:53.383668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.383956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.384265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] PATH_INFO: `/` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.385738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1160/2308] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:42:53.391134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06f8d-098b-4ba4-a8bd-9746f40130cf None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.391453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06f8d-098b-4ba4-a8bd-9746f40130cf None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.391732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06f8d-098b-4ba4-a8bd-9746f40130cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.411011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fdf06f8d-098b-4ba4-a8bd-9746f40130cf None None] MFA Rules not processed for user `78ea4fd4eacb4664af0dc24ebfcacd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:53.412216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fdf06f8d-098b-4ba4-a8bd-9746f40130cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:53.463188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1149/2309] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:53.476424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:53.478174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:53.491243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f8ba0-9f33-437c-8242-258c2b1ec7c2 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:53.495019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:53.495661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.495942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.496213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.503320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:53.504739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51a13c-776c-4f39-b8b6-a5f3c744c6bf service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:53.512961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1161/2310] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:53.597514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4f6-3cec-4dfd-825d-3f0daef6817c None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.597811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4f6-3cec-4dfd-825d-3f0daef6817c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.598112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb4f6-3cec-4dfd-825d-3f0daef6817c None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.599522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1150/2311] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:42:53.604784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.605081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.605394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.627373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] MFA Rules not processed for user `a6b4ed458d984a51829114a73534fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:53.628583 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:53.638672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1162/2312] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:53.654399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb4f6-3cec-4dfd-825d-3f0daef6817c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:53.665795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb4f6-3cec-4dfd-825d-3f0daef6817c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:53.669412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:53.670047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:53.670333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:53.670613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:53.681251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:53.682676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce7742-d97c-4e6c-bbf9-8eceeb6d6e65 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:53.694184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1151/2313] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:58.455819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:58.521894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b8bae-4f1b-4570-ae70-1956480281cb None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:58.526505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:58.527403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:58.527689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:58.527967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] PATH_INFO: `/v3/users/6ff37b01a3604cfbb9e98b0003e403eb` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:58.529281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff37b01a3604cfbb9e98b0003e403eb)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:58.531234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:58.534003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:58.675424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff37b01a3604cfbb9e98b0003e403eb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:58.676082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff37b01a3604cfbb9e98b0003e403eb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:58.676708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff37b01a3604cfbb9e98b0003e403eb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:58.686842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff37b01a3604cfbb9e98b0003e403eb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:58.696473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff37b01a3604cfbb9e98b0003e403eb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:58.772238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1163/2314] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:58 2022] DELETE /identity/v3/users/6ff37b01a3604cfbb9e98b0003e403eb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:42:58.776589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:58.777095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:58.777306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:58.796877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:58.798112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:58.861378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1152/2315] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:58.874362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:58.874362 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:58.932047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf67c-b6da-4b82-9f20-a07c760bd759 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:58.936330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:58.937052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:58.937303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:58.937582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:58.947640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:58.949131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8817dac-fc53-4995-b2b8-0f83d41edf58 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:58.959750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1164/2316] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:59.360051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:59.361497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.377273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ed7c-b383-4e23-b23e-e848280b6694 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:59.383056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:59.383973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.384306 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.384585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] PATH_INFO: `/v3/projects/220721ec418148c0b8efff4038b3e1eb` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.385978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=220721ec418148c0b8efff4038b3e1eb)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:59.387939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:59.483114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.483558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.483667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.501381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:59.502578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.534211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:59.548545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220721ec418148c0b8efff4038b3e1eb', 'request_id': 'req-ca1111aa-d782-4b3d-9162-62f9cd248c1c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:59.554906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220721ec418148c0b8efff4038b3e1eb', 'request_id': 'req-ca1111aa-d782-4b3d-9162-62f9cd248c1c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:59.558663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1165/2317] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:59.570985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:59.572355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.604184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:59.644894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220721ec418148c0b8efff4038b3e1eb', 'request_id': 'req-ca1111aa-d782-4b3d-9162-62f9cd248c1c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:59.649487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfa4a-8ce3-47f1-9bfb-3af3d84644af None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:59.661794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] The token cache is being invalidate because project 220721ec418148c0b8efff4038b3e1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:59.662085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:59.662767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-ca1111aa-d782-4b3d-9162-62f9cd248c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:59.664586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1153/2318] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:59 2022] DELETE /identity/v3/projects/220721ec418148c0b8efff4038b3e1eb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:59.671796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6f13e-7d27-4342-be67-3a249b951910 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.672446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6f13e-7d27-4342-be67-3a249b951910 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.673225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6f13e-7d27-4342-be67-3a249b951910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.695467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4cb6f13e-7d27-4342-be67-3a249b951910 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:59.696721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4cb6f13e-7d27-4342-be67-3a249b951910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.730093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:59.731630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.732555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.732930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.735805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.783183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1154/2319] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:42:59.791463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.791941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.792251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.804884 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:59.806855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:59.812506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:59.813228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:59.816408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1166/2320] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:42:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:42:59.824030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1155/2321] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:42:59.828653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:59.840128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d1c35-d27c-4a2a-9daf-e62f2e980c96 service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:59.846666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:59.847590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.848025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.848436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.849243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:42:59.857416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:59.857416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:59.865207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1167/2322] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:42:59.869321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:59.879194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e912e-fc23-4a9c-8477-175862e59c4c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:59.883920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:59.885601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:59.885601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:59.885601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:59.885859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:59.887766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:59.967099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1156/2323] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:42:59 2022] POST /identity/v3/projects => generated 397 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:42:59.971793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:59.986276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d53b21-51e5-46e6-b6d2-d14b0e400ed1 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:00.084210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:00.084969 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.085360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.085646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.087332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:00.088652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:00.100880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:00.172414 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1168/2324] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:42:59 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:43:00.173712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:00.188354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75890225-cc4a-42d6-9a03-da5fb57302ba admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:00.195658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:00.197565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.198175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.198461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.199277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:00.202008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:00.216818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1157/2325] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:43:00.220984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:00.237479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb362e-8ae6-4f5f-9366-18e8699f8327 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:00.243537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:00.244232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.244982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.245314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.246994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:00.250419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f3c43-2348-4227-83d0-7b197465315f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:00.291891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1169/2326] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:43:00.296964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:00.319959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab99187-536c-4c50-8138-2c24dc15361e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:00.331448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:00.333190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.341629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.342071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] PATH_INFO: `/v3/projects/0b6f464e31a04e28b181ba776e68723b/users/82aef31041d148699dabcd098052ff11/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.366903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6f464e31a04e28b181ba776e68723b, user_id=82aef31041d148699dabcd098052ff11, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:00.371095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:00.464311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1158/2327] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:00 2022] PUT /identity/v3/projects/0b6f464e31a04e28b181ba776e68723b/users/82aef31041d148699dabcd098052ff11/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:00.468181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.468838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.469483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.503925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] MFA Rules not processed for user `82aef31041d148699dabcd098052ff11`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:00.505364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:00.600515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1170/2328] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:00.613387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:00.615423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:00.684564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b22bf-0a25-49f9-977f-018009ed2195 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:00.688985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:00.689676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:00.689963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:00.690245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:00.699701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:00.702287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fa656-36c1-41f4-a943-8269e79fc4d6 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:00.724504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1159/2329] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:02.370916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:02.373021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:02.451423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259fd62-22f2-447f-b886-bc9380338139 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:02.456726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:02.457922 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:02.458310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:02.458700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] PATH_INFO: `/v3/users/5a85e188ea424fa2b3d5f725c0801f6b` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:02.460067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a85e188ea424fa2b3d5f725c0801f6b)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:02.462207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:03.284238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a85e188ea424fa2b3d5f725c0801f6b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:03.284863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a85e188ea424fa2b3d5f725c0801f6b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:03.285491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a85e188ea424fa2b3d5f725c0801f6b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:03.293769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a85e188ea424fa2b3d5f725c0801f6b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:03.305266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:03.311155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a85e188ea424fa2b3d5f725c0801f6b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:03.428719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1171/2330] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:43:02 2022] DELETE /identity/v3/users/5a85e188ea424fa2b3d5f725c0801f6b => generated 0 bytes in 1059 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:43:03.434151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:03.434710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:03.435468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:03.447167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:03.464821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:03.466142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:03.529120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1160/2331] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:03.542212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:03.542212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:03.615180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d05062-338a-415d-ac19-d4e35308bd4a admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:03.618879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:03.622919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:03.622919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:03.622919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:03.631366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:03.632809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1b31-f885-4652-8383-289940dd0734 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:03.654270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1172/2332] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:04.134075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.140748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.177848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb31406-9414-4532-aabc-7e69ebafc738 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.186061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.186061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.186061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.186947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] PATH_INFO: `/v3/projects/1f5afd9a249541c98259fc9da5431971` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.189242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5afd9a249541c98259fc9da5431971)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.191562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.343017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5afd9a249541c98259fc9da5431971', 'request_id': 'req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.353682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5afd9a249541c98259fc9da5431971', 'request_id': 'req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.433619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5afd9a249541c98259fc9da5431971', 'request_id': 'req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.449037 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] The token cache is being invalidate because project 1f5afd9a249541c98259fc9da5431971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:43:04.450133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.451605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dc2df1b4-98d7-4de5-bc64-82d11474bdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.454290 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1161/2333] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:43:04 2022] DELETE /identity/v3/projects/1f5afd9a249541c98259fc9da5431971 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:04.459731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.460612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.460612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.479248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:04.480461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.550259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1173/2334] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:04.561486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.561963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.562393 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.587714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:04.588909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.607949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.609231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1162/2335] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:04.618853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17589353-c051-48d6-aa0c-e1188e34202d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.624704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.625666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.626068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.626581 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.627470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:43:04.627790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.629666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.643037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1174/2336] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:43:04.647415 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.658128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f2202-23cf-42d1-8a13-28d40f3e697a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.662992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.663663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.663946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.664222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.664928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.667053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.761558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1163/2337] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:43:04 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:43:04.766131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.776291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef72cf-768a-4d1e-b811-62f87588762f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.838236 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.839227 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.839687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.839988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.840882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.843431 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.944142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1175/2338] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:43:04 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:43:04.948380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.959497 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e746d02-673a-4f71-b10a-32f0bd0be9fa admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.964046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.964701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.964988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.965307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.966073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.967659 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.978834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1164/2339] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:43:04.982717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.992904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5faa3e-2220-4708-8e2f-c83701ba7b68 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.998456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.999109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.999392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.999667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:05.000389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:05.002012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9e1c1-a5ed-4bfc-9655-9efb620a68e1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:05.003244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:05.014011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1176/2340] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:43:05.018102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:05.028046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4e5fd-f47f-4c68-b6ed-dd0cc71c8f86 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:05.032656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:05.035947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:05.035947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:05.035947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] PATH_INFO: `/v3/projects/de97a1f185f6407d9739f7ad33543abd/users/55ee6e4ceb69437bb6f3a598518261a3/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:05.048546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de97a1f185f6407d9739f7ad33543abd, user_id=55ee6e4ceb69437bb6f3a598518261a3, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:05.051232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:05.136167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1165/2341] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:05 2022] PUT /identity/v3/projects/de97a1f185f6407d9739f7ad33543abd/users/55ee6e4ceb69437bb6f3a598518261a3/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:43:05.147786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:05.148083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:05.148370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:05.179201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] MFA Rules not processed for user `55ee6e4ceb69437bb6f3a598518261a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:05.180516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:05.257137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1177/2342] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:05.267720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:05.269584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:05.312478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:05.350068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ca847-bbde-4ded-beb6-d11e90068c8e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:05.372226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:05.377539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:05.377539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:05.377539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:05.397019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:05.400011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:05.441454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1166/2343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:10.424183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:10.425700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:10.494303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c4ca-272a-4fb4-9b49-e3ecdb7fabbb None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:10.503253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:10.504355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:10.504758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:10.505197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] PATH_INFO: `/v3/users/82aef31041d148699dabcd098052ff11` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:10.506614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82aef31041d148699dabcd098052ff11)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:10.509233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:10.633615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:10.645025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc5393-21b6-4bf6-aff6-97e1c48dfa2b service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:10.650419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:10.651286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:10.651567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:10.652426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] PATH_INFO: `/v3/users/55ee6e4ceb69437bb6f3a598518261a3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:10.653202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ee6e4ceb69437bb6f3a598518261a3)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:10.655233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:10.733348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82aef31041d148699dabcd098052ff11'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.733997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82aef31041d148699dabcd098052ff11'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.734590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82aef31041d148699dabcd098052ff11'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.741580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82aef31041d148699dabcd098052ff11'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.754738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-72213c61-c5f1-4825-a17a-5a57110629d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82aef31041d148699dabcd098052ff11'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.835319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ee6e4ceb69437bb6f3a598518261a3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.835996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ee6e4ceb69437bb6f3a598518261a3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.836598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ee6e4ceb69437bb6f3a598518261a3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.844712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1178/2344] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:43:10 2022] DELETE /identity/v3/users/82aef31041d148699dabcd098052ff11 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:10.845008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ee6e4ceb69437bb6f3a598518261a3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.850763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:10.851064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:10.851375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:10.858874 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:10.860939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c82ec8f1-d7f4-492b-97a9-f752c41013a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ee6e4ceb69437bb6f3a598518261a3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:10.870665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:10.871878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:10.932044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1179/2345] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:10.943153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:10.944566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:10.946231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1167/2346] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:43:10 2022] DELETE /identity/v3/users/55ee6e4ceb69437bb6f3a598518261a3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:43:10.953394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:10.957315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:10.957315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:10.971635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:10.972852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.007269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca44ad-a65a-4f68-a80f-32b7fb8bf83e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.045541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1168/2347] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:11.057719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.059190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.062052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.062791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.063199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.063535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.073384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fed09-7666-49e9-94e6-f308f022d041 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.073558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.075053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.077853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.078505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.078795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.079076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.086512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1180/2348] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:11.089730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.091212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.101564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1169/2349] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:11.486666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.487005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.488518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.488655 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.505534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ed1f2-dd4a-4cb1-8413-4450997eb367 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.506438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15967594-7b15-4006-bcde-659efe15dd2b service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.511915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.512375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.512892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.513029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.513186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.513464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.513580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] PATH_INFO: `/v3/projects/de97a1f185f6407d9739f7ad33543abd` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.513857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] PATH_INFO: `/v3/projects/0b6f464e31a04e28b181ba776e68723b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.514715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de97a1f185f6407d9739f7ad33543abd)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.515181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6f464e31a04e28b181ba776e68723b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.516996 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.517272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.545187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:11.610828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de97a1f185f6407d9739f7ad33543abd', 'request_id': 'req-9c54366a-c7fc-4d63-9123-4b61a5b7125f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.616323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de97a1f185f6407d9739f7ad33543abd', 'request_id': 'req-9c54366a-c7fc-4d63-9123-4b61a5b7125f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.618161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:11.638999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6f464e31a04e28b181ba776e68723b', 'request_id': 'req-d765da9e-8939-41b9-93d9-59172387d16a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.644368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6f464e31a04e28b181ba776e68723b', 'request_id': 'req-d765da9e-8939-41b9-93d9-59172387d16a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.724577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de97a1f185f6407d9739f7ad33543abd', 'request_id': 'req-9c54366a-c7fc-4d63-9123-4b61a5b7125f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.738639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] The token cache is being invalidate because project de97a1f185f6407d9739f7ad33543abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:43:11.738639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.738639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-9c54366a-c7fc-4d63-9123-4b61a5b7125f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.739108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1181/2350] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:43:11 2022] DELETE /identity/v3/projects/de97a1f185f6407d9739f7ad33543abd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:11.743738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4672-bb4c-4b0b-8a64-0dd20ece3ff6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.744021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4672-bb4c-4b0b-8a64-0dd20ece3ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.744291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4672-bb4c-4b0b-8a64-0dd20ece3ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.761472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a5eb4672-bb4c-4b0b-8a64-0dd20ece3ff6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:11.763637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a5eb4672-bb4c-4b0b-8a64-0dd20ece3ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.820504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1182/2351] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:11.828058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.828347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.828625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.838534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6f464e31a04e28b181ba776e68723b', 'request_id': 'req-d765da9e-8939-41b9-93d9-59172387d16a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.847356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:11.848803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.852131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] The token cache is being invalidate because project 0b6f464e31a04e28b181ba776e68723b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:43:11.852493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.853229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d765da9e-8939-41b9-93d9-59172387d16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:11.855097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1170/2352] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:43:11 2022] DELETE /identity/v3/projects/0b6f464e31a04e28b181ba776e68723b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:11.860720 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1183/2353] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:11.861825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.862429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.862783 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.863124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.873457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cc331-78b8-425b-974a-cf57cc9276ec None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.878603 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.879262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.879542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.879816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.880545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:43:11.880852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.882187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:11.882660 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434f5cc-d8b3-4e24-acc5-de45cd9af86a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.883521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.895204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1184/2354] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:43:11.904518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1171/2355] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:11.905817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.907485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.907780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.908224 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.908758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.924040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78341c07-5cc1-4901-b289-f4b6e3493216 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.930150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.930899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.931257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.931602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.932364 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.933910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:11.934961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.935423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:11.944356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1185/2356] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:11.948617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.960486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce50b3f-d5f6-47dd-af30-6eae2ac0c114 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:11.965925 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:11.967175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:11.969398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:11.969398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:11.969398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:43:11.969398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:11.970758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:11.982842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1186/2357] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:43:11.986991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:11.997150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353c63e-c336-48c2-b0ed-459f5dfbede8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.001870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.002520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.002805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.003082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.003788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.005736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.049806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1172/2358] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:43:12.050811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.115041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529a98-6136-47b6-a23f-31a2ddaeb019 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.119778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.120702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.120984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.121293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.121973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.128121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.156047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1187/2359] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:43:11 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:43:12.159943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.220967 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b0eda-3719-4252-82f7-4575e0123007 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.225701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.226360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.226663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.227024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.227716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.229668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.252455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1173/2360] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:43:12 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:43:12.255495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.265647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe043abe-9134-4486-a86e-2d7abfd3a51b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.270277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.270912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.271196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.271475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.272192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.273887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.284224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1174/2361] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:43:12.287795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.297349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b84a0f-2e9a-4a83-b0a7-e16fed08edf9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.302178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.302788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.303073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.303350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.304064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.305646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.315827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1175/2362] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:43:12.319452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.328985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1d1ad-85d1-4ba7-b1c8-28aad1da2af7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.333605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.334431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.334714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.334987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] PATH_INFO: `/v3/projects/1f15caa06dac4f2b952ff64f9e9e74ca/users/9ac06a23a9b84ec38e0712bad9046762/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.339172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1188/2363] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:43:12 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:43:12.346991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.350770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f15caa06dac4f2b952ff64f9e9e74ca, user_id=9ac06a23a9b84ec38e0712bad9046762, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.353457 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea099-c828-49d5-b08f-0ae730e3c06f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.357049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d2baf-63f5-4aa2-a687-8599ea8137ac admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.362293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.362950 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.363233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.363509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.364317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.365938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.376767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1189/2364] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:43:12.380772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.386652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:12.391051 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706d62f-2707-4af4-a7c0-83e212b36f95 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.395726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.396336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.396656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.396934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.397671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.399200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.410301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1190/2365] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:43:12.414197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.424039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431825b-d247-4486-9cdd-1ded25b16316 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.428791 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1176/2366] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:12 2022] PUT /identity/v3/projects/1f15caa06dac4f2b952ff64f9e9e74ca/users/9ac06a23a9b84ec38e0712bad9046762/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:43:12.434040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.434337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.435423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.456045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:12.457298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:12.475170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:12.490010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.490896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.491175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.491454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] PATH_INFO: `/v3/projects/c361a671e3fc43c8a536f6526dc530b1/users/917bfc7c68384536a6945af83308f789/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.506957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c361a671e3fc43c8a536f6526dc530b1, user_id=917bfc7c68384536a6945af83308f789, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.509889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f930129-81cb-4248-93e4-0dab9f790471 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.516845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1177/2367] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:12.526568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.528051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:12.571109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1191/2368] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:12 2022] PUT /identity/v3/projects/c361a671e3fc43c8a536f6526dc530b1/users/917bfc7c68384536a6945af83308f789/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:43:12.576244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.576541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.576821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.585878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf17f6-4f26-4f4e-b209-fbc7cad3a0c4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.598168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:12.599484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:12.637028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.637841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.638159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.638446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.659148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1192/2369] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:12.670589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:12.680053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ff8ea-4321-455b-a15b-4386c68d749d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:12.683976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:12.684692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:12.685115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:12.685418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:12.694670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.696127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e7bd7-ebec-49a6-b542-511fbad431bd service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.701616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:12.703054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:12.706202 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1193/2370] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:12.713087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1178/2371] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:43:14.275712 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75a2c-dae7-4608-9759-6e2c807f0e28 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:14.276257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75a2c-dae7-4608-9759-6e2c807f0e28 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:14.276257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75a2c-dae7-4608-9759-6e2c807f0e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:14.304264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-faa75a2c-dae7-4608-9759-6e2c807f0e28 None None] MFA Rules not processed for user `9ac06a23a9b84ec38e0712bad9046762`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:14.305541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-faa75a2c-dae7-4608-9759-6e2c807f0e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:14.366806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:14.382337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1194/2372] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:14.395503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:14.397639 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:14.452406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:14.483734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df24d-6657-4048-9a24-b124a246735a service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:14.489194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:14.493527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:14.493527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:14.493527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:14.501499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:14.503131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e6c8c-b540-45f8-abc0-3521178292ce service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:14.526974 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1179/2373] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:16.478296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:16.478868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:16.478868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:16.502393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] MFA Rules not processed for user `9ac06a23a9b84ec38e0712bad9046762`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:16.503758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:16.515190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1195/2374] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:18.766002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:18.766543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:18.766543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:18.795808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] MFA Rules not processed for user `917bfc7c68384536a6945af83308f789`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:18.797085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:18.873292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1180/2375] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:18.877260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:18.894232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12790c-9cc9-47f4-913b-4a1ca05d58e7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:18.899110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:18.900043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:18.900549 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:18.901310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:18.902815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:18.905750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.025762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1196/2376] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:43:18 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:43:19.029929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:19.057932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a0a0e-18b7-4e27-85c3-445d8ff7cf22 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:19.173736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:19.174442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:19.174733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:19.175161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:19.175871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:19.177906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.273619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1181/2377] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:43:19 2022] POST /identity/v3/users => generated 295 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:43:19.277571 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:19.288652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bd44-016f-4a8e-b1d2-35f47b68e809 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:19.294074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:19.294846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:19.295131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:19.295407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:19.296146 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:19.297820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.312562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1197/2378] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:43:19.316932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:19.330143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b884abd-0451-4c80-8ee6-e3ddadb8db4f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:19.334966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:19.335878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:19.336160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:19.336434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] PATH_INFO: `/v3/projects/1dfe50850f27465da257fe324ca5067e/users/0a37cca180604534860037a2641b117e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:19.355101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfe50850f27465da257fe324ca5067e, user_id=0a37cca180604534860037a2641b117e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:19.358136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.462448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1182/2379] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:19 2022] PUT /identity/v3/projects/1dfe50850f27465da257fe324ca5067e/users/0a37cca180604534860037a2641b117e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:19.466477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:19.481117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180bbe-13c0-445d-8bd9-327fec03e469 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:19.572734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:19.573716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:19.574193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:19.574676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:19.575812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:19.581929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53a49-8f5e-4405-914a-1b3d437a8e17 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.593679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1198/2380] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:43:19.597672 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:19.628296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9a7cf-5c2c-4a9f-bff8-9261e92d8d61 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:19.633916 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:19.634825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:19.635110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:19.635388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] PATH_INFO: `/v3/projects/1dfe50850f27465da257fe324ca5067e/users/0a37cca180604534860037a2641b117e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:19.637490 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfe50850f27465da257fe324ca5067e, user_id=0a37cca180604534860037a2641b117e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:19.640612 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:19.655329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1183/2381] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:43:19 2022] PUT /identity/v3/projects/1dfe50850f27465da257fe324ca5067e/users/0a37cca180604534860037a2641b117e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:23.707318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ef12d-3567-499c-84c1-7c23bdc6ad29 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:23.708561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ef12d-3567-499c-84c1-7c23bdc6ad29 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:23.709055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ef12d-3567-499c-84c1-7c23bdc6ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:23.728688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1e4ef12d-3567-499c-84c1-7c23bdc6ad29 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:23.730338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1e4ef12d-3567-499c-84c1-7c23bdc6ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:23.855028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1199/2382] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:23.877736 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:23.881917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:23.898771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:23.964800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e218-07f6-4285-a9dc-718d43fde959 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:23.972496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:23.973263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:23.974512 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:23.974820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:23.988347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:23.989538 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:23.999257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1184/2383] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:43:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:43:26.715651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7c8a-b1e2-4291-8c20-d82056b9026d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:26.716678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7c8a-b1e2-4291-8c20-d82056b9026d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:26.716787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed7c8a-b1e2-4291-8c20-d82056b9026d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:26.744740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2aed7c8a-b1e2-4291-8c20-d82056b9026d None None] MFA Rules not processed for user `0a37cca180604534860037a2641b117e`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:26.746506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2aed7c8a-b1e2-4291-8c20-d82056b9026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:26.807131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:26.838081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1200/2384] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:26.851990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:26.854221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:26.929909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857808-4587-49c8-b2c6-20b15b9edb19 service swift] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:26.935996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:26.936650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:26.936935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:26.937245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:26.998803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:27.000299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d486b-42d0-4757-97d0-5470113f167f service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:27.009988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1185/2385] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:36.863044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=9 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:37.005537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=82 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:41.662622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeab35-756d-46c9-a0df-ebd6ef70ad54 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:41.663101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeab35-756d-46c9-a0df-ebd6ef70ad54 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:41.663208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeab35-756d-46c9-a0df-ebd6ef70ad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:41.682625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-edeeab35-756d-46c9-a0df-ebd6ef70ad54 None None] MFA Rules not processed for user `9ac06a23a9b84ec38e0712bad9046762`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:41.683849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-edeeab35-756d-46c9-a0df-ebd6ef70ad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:41.741498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1201/2386] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:50.554960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a425d-ff61-4351-a715-2aa84b8ee5de None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:50.555447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a425d-ff61-4351-a715-2aa84b8ee5de None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:50.555562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a425d-ff61-4351-a715-2aa84b8ee5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:50.572820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6a7a425d-ff61-4351-a715-2aa84b8ee5de None None] MFA Rules not processed for user `917bfc7c68384536a6945af83308f789`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:50.574071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6a7a425d-ff61-4351-a715-2aa84b8ee5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:50.581628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1186/2387] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:43:51.734724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=25 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:58.599164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:58.599790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:58.599790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:58.617337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] MFA Rules not processed for user `97f1b1e52a984639a504441e3701b981`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:58.618537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:58.676383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1202/2388] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:44:00.567984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:08.669420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=25 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:13.047374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:13.050788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:13.050788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:13.066570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:13.067784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:13.075170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1187/2389] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:13.097422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:13.109368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642a4b3-f8c6-4c73-a991-b0b4d51357ab None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:13.115031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:13.115675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:13.115962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:13.116244 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:13.124506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:13.126166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:13.134225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1203/2390] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:44:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:44:17.127406 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:17.188224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c91c43-dfb7-4d2c-8c96-1af1e1de048b None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:17.192064 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:17.192865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:17.193109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:17.193429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:17.195254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:17.210157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:17.214246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:17.229730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1188/2391] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:18.309463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:18.381658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0bad3-3def-400b-a86e-a4410ff229f0 service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:18.385648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:18.386376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:18.386799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:18.387028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:18.389007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:18.404752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:18.407791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6324b7-1ee3-4799-a035-9b88a1c2b25d service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:18.455460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1204/2392] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:21.321072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:21.331124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411fa2e-46f6-414a-b422-f55259484670 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:21.334860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:21.335504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:21.335788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:21.336066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:21.345032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:21.346477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:21.355839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1189/2393] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:28.435540 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=32 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:31.351298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.352316 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.352631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=35 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:31.353040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.371068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:31.372312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.389808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1205/2394] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:31.398902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.413466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c373-5bbf-477a-81d8-c44cc148bc84 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.418826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.419472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.419754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.420033 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.436146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.437898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8717f-dfc2-44c8-abc0-a086b1829cf6 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.453338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1190/2395] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:44:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:44:31.957123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.966696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e168a-e684-41ef-9efe-6a9ab2ff0f6a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.971612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.971612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.971612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.971612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.973477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:32.040990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.042456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.051822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1206/2396] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:33.809712 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:33.810205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:33.810303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:33.828157 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] MFA Rules not processed for user `a760d437b597441fa9a4595995c5bc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:33.829523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:33.836496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1191/2397] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:34.565276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:34.566735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:34.658773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8a4a-4e83-47d4-a869-f5c24d252b92 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:34.664094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:34.664907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:34.665332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:34.665494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:34.667674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:34.680614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:34.682288 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:34.692810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1207/2398] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:34.773169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:34.849534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2145c1-e9cc-46d1-83b1-ad2fe25dcca9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:34.854294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:34.855531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:34.855654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:34.855859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] PATH_INFO: `/v3/users/9ac06a23a9b84ec38e0712bad9046762` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:34.857058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac06a23a9b84ec38e0712bad9046762)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:34.859042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:35.068460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac06a23a9b84ec38e0712bad9046762'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.069111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac06a23a9b84ec38e0712bad9046762'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.069766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac06a23a9b84ec38e0712bad9046762'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.076869 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac06a23a9b84ec38e0712bad9046762'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.089314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b30cb6e6-06cb-4a47-a994-126b123617b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac06a23a9b84ec38e0712bad9046762'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.173254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1192/2399] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:44:34 2022] DELETE /identity/v3/users/9ac06a23a9b84ec38e0712bad9046762 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:35.390966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:35.402208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7d87-2114-4595-bf6c-fa2c389889d7 service placement] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:35.461301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:35.467292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:35.467292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:35.467292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] PATH_INFO: `/v3/projects/1f15caa06dac4f2b952ff64f9e9e74ca` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:35.467292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f15caa06dac4f2b952ff64f9e9e74ca)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:35.467292 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:35.491312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:35.626943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f15caa06dac4f2b952ff64f9e9e74ca', 'request_id': 'req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.633105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f15caa06dac4f2b952ff64f9e9e74ca', 'request_id': 'req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.726862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f15caa06dac4f2b952ff64f9e9e74ca', 'request_id': 'req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.739075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] The token cache is being invalidate because project 1f15caa06dac4f2b952ff64f9e9e74ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:44:35.742718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.742718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ea2a9fa5-2236-4af0-aad5-1f71d7180db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:35.742718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1208/2400] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:44:35 2022] DELETE /identity/v3/projects/1f15caa06dac4f2b952ff64f9e9e74ca => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:35.746656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:35.746958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:35.747236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:35.765782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:35.767565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:35.817909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:35.826018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1193/2401] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:35.834370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:35.834661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:35.834945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:35.853042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:35.854267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:35.862193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1209/2402] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:35.866352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:35.879057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c6520-5046-48c6-a29d-eafba104821a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:35.884136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:35.884859 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:35.885128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:35.885447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:35.886236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:44:35.887148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:35.888202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:35.900678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1194/2403] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:44:35.904993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:35.914841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21469f2f-1b9b-4e37-949e-23c60ffc672c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:35.920095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:35.920746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:35.921030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:35.921374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:35.922275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:35.924858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.003657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1210/2404] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:44:35 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:44:36.007632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:36.017902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62245de-07b9-4dd3-88aa-917d622d8b6b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:36.074435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:36.075126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.075412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.075692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.076392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:36.078337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.162109 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1195/2405] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:44:36 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:44:36.166175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:36.177356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bffed-d879-4452-919f-ef0d904e5e96 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:36.181995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:36.182659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.182983 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.183262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.183990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:36.185624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.206351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1211/2406] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:36.208917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:36.219853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39a0a-6929-4041-bf99-97030498b627 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:36.225313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:36.226114 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.226504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.227014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.228118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:36.231345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920174e9-a89b-48d3-a4b4-a5b13d45eccc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.247379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1196/2407] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:44:36.250234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:36.262401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826893d-8a1c-4cdb-8307-333c843de744 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:36.267531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:36.268425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.268708 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.269110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] PATH_INFO: `/v3/projects/f5e6f8aec4ba48789e2b40eb2196e73c/users/cbd982cea06a4e9ebecf3eee2590e8d8/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.288155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e6f8aec4ba48789e2b40eb2196e73c, user_id=cbd982cea06a4e9ebecf3eee2590e8d8, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:36.290842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.392247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1212/2408] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:44:36 2022] PUT /identity/v3/projects/f5e6f8aec4ba48789e2b40eb2196e73c/users/cbd982cea06a4e9ebecf3eee2590e8d8/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:36.403862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba912-cad8-4211-ae86-e44174e6f1df None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.404150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba912-cad8-4211-ae86-e44174e6f1df None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.404432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba912-cad8-4211-ae86-e44174e6f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.431932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-64cba912-cad8-4211-ae86-e44174e6f1df None None] MFA Rules not processed for user `cbd982cea06a4e9ebecf3eee2590e8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:36.433058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-64cba912-cad8-4211-ae86-e44174e6f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:36.499143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1197/2409] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:36.509811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:36.511526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:36.569561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90f7d3-cf54-4f90-8c55-cdafd1311adc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:36.573307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:36.574445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:36.574445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:36.574628 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:36.583052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:36.584466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaded66-23ba-4cb0-b851-02b4a9b5f086 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:36.593865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1213/2410] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:38.394283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:38.394799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:38.394949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:38.412735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] MFA Rules not processed for user `cbd982cea06a4e9ebecf3eee2590e8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:38.413975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:38.420886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1198/2411] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:41.621253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:41.622486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:41.622486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:41.662918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:41.664892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:41.730031 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:41.740716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1214/2412] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:41.754679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:41.756148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:41.810292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:41.831488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ff1d-e475-4d98-af56-be1fbbe41c76 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:41.836554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:41.837243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:41.837530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:41.837804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:41.846716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:41.848300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53232db-0c9a-427b-880c-f670638a5383 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:41.858586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1199/2413] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:44:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:44:48.094704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:48.096412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:48.160877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0905c3-8cc6-485f-afba-8c9ae87963c5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:48.170402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:48.170402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:48.170402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:48.170402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:48.171775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:48.184280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:48.186637 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1816628-bfd8-4f0a-b189-1d3f09aa6e96 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:48.201544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1215/2414] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:51.228862 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:51.230018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:51.230353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] PATH_INFO: `/` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:51.232116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1200/2415] 10.222.0.30 () {54 vars in 921 bytes} [Tue Jul 12 14:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:44:51.243741 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:51.245247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:51.245856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:51.271987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] MFA Rules not processed for user `e40592fc25234d65a0be99d3d29869cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:51.273624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:51.339545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1216/2416] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Jul 12 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:51.354619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:51.357752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:51.363407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:51.365143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:51.378444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4c189-8c06-4ea9-8f72-84af511de2be None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:51.383688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:51.384408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:51.384633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:51.384910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:51.394326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:51.396185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:51.421412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1201/2417] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:51.459458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bdac5-0fb6-4301-9225-153d164c0f22 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:51.465435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:51.466345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:51.466626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:51.466902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] PATH_INFO: `/v3/users/92fed4b668e246f6b1b89a888c34f107` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:51.468188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92fed4b668e246f6b1b89a888c34f107)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:51.471006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:51.652504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:51.662867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92fed4b668e246f6b1b89a888c34f107'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:51.664141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92fed4b668e246f6b1b89a888c34f107'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:51.664823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92fed4b668e246f6b1b89a888c34f107'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:51.672476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92fed4b668e246f6b1b89a888c34f107'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:51.686280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92fed4b668e246f6b1b89a888c34f107'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:51.790278 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1217/2418] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:44:51 2022] DELETE /identity/v3/users/92fed4b668e246f6b1b89a888c34f107 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:52.084747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:52.086175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:52.130900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:52.132557 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:52.176821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d624094-9fb7-4765-b053-ce1bd1b32db5 service placement] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:52.212089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8207-dc9a-4d13-a1c3-cb2b29695cf6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:52.217409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:52.222219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:52.222219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:52.222219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] PATH_INFO: `/v3/projects/47cd27ad4219485cae0d8d85a2823948` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:52.222219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47cd27ad4219485cae0d8d85a2823948)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:52.222219 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:52.251884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:52.252539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:52.253006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:52.253330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:52.255193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:52.267969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:52.269673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:52.280466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1202/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:52.357776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47cd27ad4219485cae0d8d85a2823948', 'request_id': 'req-cbc8506e-a5c9-4ab7-8566-5b608d678526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.363709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47cd27ad4219485cae0d8d85a2823948', 'request_id': 'req-cbc8506e-a5c9-4ab7-8566-5b608d678526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.447389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47cd27ad4219485cae0d8d85a2823948', 'request_id': 'req-cbc8506e-a5c9-4ab7-8566-5b608d678526'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.459693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] The token cache is being invalidate because project 47cd27ad4219485cae0d8d85a2823948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:44:52.460114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.460929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.464476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1218/2420] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:44:52 2022] DELETE /identity/v3/projects/47cd27ad4219485cae0d8d85a2823948 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:52.468439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:52.480370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615f60c-2c99-436c-a116-626e8210c928 service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:52.542359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:52.559241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:52.560429 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:52.560710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:52.561172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] PATH_INFO: `/v3/users/d66e80586c0747aabe2e4e9a0cf8db86` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:52.562454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66e80586c0747aabe2e4e9a0cf8db86)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:52.565042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:52.750327 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66e80586c0747aabe2e4e9a0cf8db86'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.751686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66e80586c0747aabe2e4e9a0cf8db86'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.752381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66e80586c0747aabe2e4e9a0cf8db86'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.760179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66e80586c0747aabe2e4e9a0cf8db86'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.774338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-42d21b1b-4c60-455b-8a32-ffdcebeac27a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66e80586c0747aabe2e4e9a0cf8db86'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:52.859370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1203/2421] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:44:52 2022] DELETE /identity/v3/users/d66e80586c0747aabe2e4e9a0cf8db86 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:53.164214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:53.165866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:53.181395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8506e-a5c9-4ab7-8566-5b608d678526 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:53.238736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:53.255513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:53.256406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.256689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.257144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] PATH_INFO: `/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.258577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a850cf48283f46bbac5b5a8e4f1a8335)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:53.260663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:53.413220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a850cf48283f46bbac5b5a8e4f1a8335', 'request_id': 'req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:53.418528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a850cf48283f46bbac5b5a8e4f1a8335', 'request_id': 'req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:53.521815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a850cf48283f46bbac5b5a8e4f1a8335', 'request_id': 'req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:53.533201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] The token cache is being invalidate because project a850cf48283f46bbac5b5a8e4f1a8335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:44:53.533495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:53.534106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bf1fb37f-c2cd-4538-9fa9-463deebf91ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:53.535704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1219/2422] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:44:53 2022] DELETE /identity/v3/projects/a850cf48283f46bbac5b5a8e4f1a8335 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:53.540900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.541411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.541513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.563799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:53.563799 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:53.623904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1204/2423] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:53.632830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.633122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.633434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.652894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:53.654127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:53.662243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1220/2424] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:53.666584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:53.676513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bb29e-ba42-4b33-9391-697e293f6590 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:53.681136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:53.681815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.682094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.682367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.683106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:44:53.683384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:53.684981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:53.702724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1205/2425] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:44:53.705136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:53.714775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984ab20-24ab-4c56-8953-c74d82e73d71 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:53.719586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:53.720240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.720525 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.720807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.721543 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:53.723470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:53.829327 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1221/2426] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:44:53 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:44:53.831175 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:53.841667 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cfd00-1749-47b0-bf04-ee3439d6c301 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:53.890485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:53.897470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:53.898174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:53.898464 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:53.898708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:53.899405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:53.901366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.002545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1206/2427] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:44:53 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:44:54.006675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:54.017149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925933bf-d773-4333-b568-4bbfe2b1c18e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:54.027128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:54.027865 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.028099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.028382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.029125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:54.030763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.041702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1222/2428] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:54.045695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:54.056377 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf8589-d8c5-4dab-bf71-3ddd62311483 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:54.061182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:54.061858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.062146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.062479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.063221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:54.065374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a10b-5bb1-400c-82f1-cb72367e0565 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.076247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1207/2429] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:54.080620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:54.090576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b51ed-9abc-4b70-abdf-5fda8507cbfa admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:54.095184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:54.096058 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.096345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.096626 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] PATH_INFO: `/v3/projects/8fa20247a7af46d0b9660d4cd026f552/users/df735fc7405643f285f529ddfa611064/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.111918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa20247a7af46d0b9660d4cd026f552, user_id=df735fc7405643f285f529ddfa611064, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:54.114710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.206461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1223/2430] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:44:54 2022] PUT /identity/v3/projects/8fa20247a7af46d0b9660d4cd026f552/users/df735fc7405643f285f529ddfa611064/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:54.211470 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.211856 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.212144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.229582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:54.230839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:54.291412 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1208/2431] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:54.301294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:54.302973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:54.357826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cb7d6-a833-4dc7-a2ef-6fc995dc36b2 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:54.362091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:54.362724 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.363005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.363280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.372103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:54.373560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.383104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1224/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:54.738530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:54.739947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:54.802294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc272a-f91a-45c4-ac77-e1016f11743e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:54.805951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:54.806604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:54.806887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:54.807173 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:54.808980 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:54.869144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:54.870622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac982b-ea7e-44d3-957a-26faf14cb09b service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:54.881874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1209/2433] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:57.330824 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:57.355034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e95f18-e8a5-4097-9e96-86090561820b service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:57.363298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:57.364266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:57.365743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:57.366131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:57.368756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:57.386563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:57.468158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:57.470285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:57.481313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1225/2434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:58.654181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933f34-d2ae-4041-a8ee-3c24ea96ce4b None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:58.654729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933f34-d2ae-4041-a8ee-3c24ea96ce4b None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:58.654729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933f34-d2ae-4041-a8ee-3c24ea96ce4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:58.675100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-55933f34-d2ae-4041-a8ee-3c24ea96ce4b None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:58.676383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-55933f34-d2ae-4041-a8ee-3c24ea96ce4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:58.706413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:58.742384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1210/2435] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:58.757928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:58.760078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:58.837609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0ead-e6aa-4ccb-9de8-298bba63ab28 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:58.842798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:58.843460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:58.843739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:58.844016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:58.852926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:58.854555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:58.863492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1226/2436] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:44:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:45:00.095529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2c6-65ed-4304-ae56-8e4daca442ab None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.096421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2c6-65ed-4304-ae56-8e4daca442ab None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.096719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2c6-65ed-4304-ae56-8e4daca442ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.115914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-db79e2c6-65ed-4304-ae56-8e4daca442ab None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:00.117251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-db79e2c6-65ed-4304-ae56-8e4daca442ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:00.133572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1211/2437] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:00.143602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.157117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf4b07-36f9-412b-ad65-b5f2b8ad6765 service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.164345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:00.166153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.166454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.166736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.177582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:00.179677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:00.190194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1227/2438] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:45:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:45:00.388030 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.388298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.388573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.418964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] MFA Rules not processed for user `df735fc7405643f285f529ddfa611064`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:00.420187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:00.490794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1212/2439] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:00.495351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.505989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c053f4e-726b-4969-ac88-75764debe696 service swift] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.510488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:00.511129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.511411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.511688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.512384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:00.514358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:00.609532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1228/2440] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:00 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:00.613645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.627748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d62ed-4698-44c9-91fa-089af28fbf6e None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.710281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:00.718848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.718848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.718848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.718848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:00.718848 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04515785-2c79-408f-9b15-d5e0b8b3d61b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:00.799343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1213/2441] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:00 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:00.801089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.819469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.819758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.820046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.820867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7f7bf-9e4f-414e-8c22-20fc4306c7c4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.826179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:00.827093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.827594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.828033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.829618 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:00.831391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:00.842219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:00.843795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:00.844286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1229/2442] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:00.848472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.858988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85984d-c5f8-4ff3-8a62-15b1f1dd2634 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.863852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:00.864768 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:00.865048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:00.865421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] PATH_INFO: `/v3/projects/f699e58f71d24bb7b4a700b3b3586180/users/2da99a3d46c34811b3d895f24e69b3a8/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:00.882721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f699e58f71d24bb7b4a700b3b3586180, user_id=2da99a3d46c34811b3d895f24e69b3a8, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:00.885398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:00.890196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:00.903787 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1214/2443] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:00.916326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.950452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1230/2444] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:00 2022] PUT /identity/v3/projects/f699e58f71d24bb7b4a700b3b3586180/users/2da99a3d46c34811b3d895f24e69b3a8/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:00.954383 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:00.985680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698db2ff-ab0e-462e-8eae-4cae8f714c9a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:00.991090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:01.022266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65621-330b-4d15-bea0-4ab1ad64cfe5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:01.027690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:01.028418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:01.028756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:01.029028 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:01.029837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:01.031499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:01.044442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1231/2445] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:45:01.049329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:01.054692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:01.055401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:01.055638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:01.055917 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:01.059837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f06df-fc0b-4867-973b-71f80330bbf9 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:01.065061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:01.066522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:01.066932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:01.067218 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] PATH_INFO: `/v3/projects/f699e58f71d24bb7b4a700b3b3586180/users/2da99a3d46c34811b3d895f24e69b3a8/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:01.069317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f699e58f71d24bb7b4a700b3b3586180, user_id=2da99a3d46c34811b3d895f24e69b3a8, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:01.071979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bd1f5-9da6-44a8-b0f4-04e942d0cf5d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:01.083220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1232/2446] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:01 2022] PUT /identity/v3/projects/f699e58f71d24bb7b4a700b3b3586180/users/2da99a3d46c34811b3d895f24e69b3a8/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:01.118154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:01.119640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:01.128444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1215/2447] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:45:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:45:01.724651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:01.725566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:01.726012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:01.748153 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:01.750958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:01.823426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1233/2448] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:01.837226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:01.851952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf8ff3-7a9c-44e5-8e41-5fc013be9f18 service swift] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:01.911048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:01.942480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:01.943657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:01.944136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:01.944641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:01.954601 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:01.956544 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1d64a-dddf-4eca-9876-e8770ab00318 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:01.966172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1216/2449] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:45:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:45:04.924999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:04.927201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:05.045885 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b32d-75b1-4767-9ed4-900c08037234 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:05.051771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:05.052912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:05.053363 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:05.053743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] PATH_INFO: `/v3/users/cbd982cea06a4e9ebecf3eee2590e8d8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:05.056397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd982cea06a4e9ebecf3eee2590e8d8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:05.058962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:05.278531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd982cea06a4e9ebecf3eee2590e8d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:05.279082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd982cea06a4e9ebecf3eee2590e8d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:05.279746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd982cea06a4e9ebecf3eee2590e8d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:05.301982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd982cea06a4e9ebecf3eee2590e8d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:05.327140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:05.337023 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd982cea06a4e9ebecf3eee2590e8d8'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:05.429837 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1234/2450] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:04 2022] DELETE /identity/v3/users/cbd982cea06a4e9ebecf3eee2590e8d8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:05.433699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:05.434186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:05.434273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:05.512641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:05.515498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:05.672896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1217/2451] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:05.682360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:05.687645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:05.787070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8706a1-55e0-411b-ba77-18ae19bbac2e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:05.791869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:05.793192 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:05.793876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:05.794589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:05.806358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:05.809930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:05.834294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1235/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:06.218605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:06.220098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:06.243677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b7fa0-10e0-4e61-b076-5558e5a86083 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:06.262099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:06.264264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:06.264264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:06.264264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] PATH_INFO: `/v3/projects/f5e6f8aec4ba48789e2b40eb2196e73c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:06.267496 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e6f8aec4ba48789e2b40eb2196e73c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:06.276095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:06.287133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:06.290330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:06.463901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24874-c53d-48e6-afcf-3f0800a82332 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:06.467185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:06.467697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:06.467895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:06.468246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:06.475158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:06.493634 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:06.495400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a9fa5-ef50-4510-aec5-18d4c2a00115 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:06.528569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1236/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:45:06.580408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e6f8aec4ba48789e2b40eb2196e73c', 'request_id': 'req-b6572196-7db0-40fe-aa71-af6633b5300f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:06.593376 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e6f8aec4ba48789e2b40eb2196e73c', 'request_id': 'req-b6572196-7db0-40fe-aa71-af6633b5300f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:06.706719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e6f8aec4ba48789e2b40eb2196e73c', 'request_id': 'req-b6572196-7db0-40fe-aa71-af6633b5300f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:06.724165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] The token cache is being invalidate because project f5e6f8aec4ba48789e2b40eb2196e73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:06.724702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:06.725758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:06.741202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1218/2454] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:06 2022] DELETE /identity/v3/projects/f5e6f8aec4ba48789e2b40eb2196e73c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:09.530426 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a669a-f643-46c8-ab97-8f44ade408d3 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:09.530955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a669a-f643-46c8-ab97-8f44ade408d3 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:09.531070 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a669a-f643-46c8-ab97-8f44ade408d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:09.559799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0f6a669a-f643-46c8-ab97-8f44ade408d3 None None] MFA Rules not processed for user `2da99a3d46c34811b3d895f24e69b3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:09.561088 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0f6a669a-f643-46c8-ab97-8f44ade408d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:09.631947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:09.651308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1237/2455] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:09.666311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:09.667776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:09.726484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:09.752506 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572196-7db0-40fe-aa71-af6633b5300f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:09.758558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:09.759214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:09.759498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:09.759780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:09.762294 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:09.825176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:09.826810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:09.836936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1219/2456] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:11.961780 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e294-b73d-47a7-8517-107c6a972f94 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:11.962624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e294-b73d-47a7-8517-107c6a972f94 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:11.962907 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338e294-b73d-47a7-8517-107c6a972f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:11.982381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5338e294-b73d-47a7-8517-107c6a972f94 None None] MFA Rules not processed for user `8f118c0ba8494e6e9f5a4488ebf8d220`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:11.983794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5338e294-b73d-47a7-8517-107c6a972f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.043441 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1238/2457] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:12.460830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.463913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.552083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18241a-6afa-4f02-9c6c-e121a1467d06 service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.557022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.557720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.558168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.558453 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.560964 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.574493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.576380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.587494 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1220/2458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:13.106533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.106994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.107290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.141577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] MFA Rules not processed for user `df735fc7405643f285f529ddfa611064`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:13.143175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:13.167171 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1239/2459] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:22.582170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=60 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:23.127973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=32 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:24.378693 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.387241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ef311-47a5-41e3-bc5c-6db504eb7a5b service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.390885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.391523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.391804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.392078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.393872 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:24.462810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.464562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.476372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1221/2460] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:25.918528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.920044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:25.937333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef61a7-d86a-4008-b3b8-f0512f0233cc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.943307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.943991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.944910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.944910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.946562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:25.960471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.962198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.974846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1240/2461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:28.138341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.139768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:28.152189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc164a8-863b-4ff6-bf6b-6c5283cfc5a6 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.157360 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.158235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.158522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.158809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] PATH_INFO: `/v3/users/97f1b1e52a984639a504441e3701b981` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.160077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f1b1e52a984639a504441e3701b981)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.162059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.307128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f1b1e52a984639a504441e3701b981'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.307833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f1b1e52a984639a504441e3701b981'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.308463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f1b1e52a984639a504441e3701b981'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.316263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f1b1e52a984639a504441e3701b981'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.329049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-fef165c7-b721-4490-8d9e-69ad42676e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f1b1e52a984639a504441e3701b981'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.391668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1222/2462] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:28 2022] DELETE /identity/v3/users/97f1b1e52a984639a504441e3701b981 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:28.636044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.649042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea3a8-bbba-4715-ad45-c364fe3782a4 service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.706941 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.707902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.708068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.708341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] PATH_INFO: `/v3/projects/fe2b01873d92453599763e5523a862f8` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.709569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2b01873d92453599763e5523a862f8)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.711490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.863593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2b01873d92453599763e5523a862f8', 'request_id': 'req-fdf58ade-8606-4109-95bc-a8fde885a7fa'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.869134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2b01873d92453599763e5523a862f8', 'request_id': 'req-fdf58ade-8606-4109-95bc-a8fde885a7fa'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.973640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2b01873d92453599763e5523a862f8', 'request_id': 'req-fdf58ade-8606-4109-95bc-a8fde885a7fa'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.985295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] The token cache is being invalidate because project fe2b01873d92453599763e5523a862f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:28.985586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.986212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-fdf58ade-8606-4109-95bc-a8fde885a7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.987818 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1241/2463] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:28 2022] DELETE /identity/v3/projects/fe2b01873d92453599763e5523a862f8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:28.993888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.994184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.994467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.012837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:29.014077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.073527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1223/2464] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:29.082117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.082412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.082813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.101687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:29.102886 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.111114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1242/2465] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:29.115399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.126021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5821f2-9273-476f-87e7-7ffc11201de0 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.130466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.131209 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.131420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.131703 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.132428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:29.132741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.134425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.135981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:29.148038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1224/2466] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:29.152606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.163110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c934-aa35-4eb7-8ab4-2197fd1357e6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.168168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.168828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.169115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.169430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.170214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.172117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.290982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1243/2467] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:29.292062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.302785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208601f-21b3-4b0d-ac73-86b88f5e0693 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.364354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.365073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.365341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.365619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.366303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.368221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.470152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1225/2468] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:29.670328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.670328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd74312-0f9b-4560-b41a-d7acaa939c57 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.670328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.670328 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.671114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.671114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.671114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.671114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.671114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1244/2469] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:29.671486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.671486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a226a02-81a8-47eb-833d-9277678630f9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.671486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.671486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.671808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.671808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.671808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.671808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa6301-c3a5-4613-8dab-ea06d720eb51 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.671808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1226/2470] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:29.672187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.672187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4932312-31e5-4d72-b0cb-2ad00e54be5a admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.672187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.672187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.672501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.672501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] PATH_INFO: `/v3/projects/a285734c876649f89a753f86795e24c4/users/3ca554e3e0604d619934ef9c00b3aa9b/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.672501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a285734c876649f89a753f86795e24c4, user_id=3ca554e3e0604d619934ef9c00b3aa9b, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.672501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.760002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1245/2471] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:29 2022] PUT /identity/v3/projects/a285734c876649f89a753f86795e24c4/users/3ca554e3e0604d619934ef9c00b3aa9b/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:29.764255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.764700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.764963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.799752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] MFA Rules not processed for user `3ca554e3e0604d619934ef9c00b3aa9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:29.800979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.872144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1227/2472] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:29.885852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.887586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.903776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:29.965703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b949ff-3efe-45d6-bcaf-97630345b003 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.971482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.972310 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.972938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.973422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.981813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.983561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3010-1d1e-4cff-976b-b7f90913bf78 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.992486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1246/2473] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:45:33.386413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:33.449449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646794d5-5fc9-4824-9b33-ef9bfb0af1a5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:33.453977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:33.454912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:33.455200 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:33.455977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] PATH_INFO: `/v3/users/3ca554e3e0604d619934ef9c00b3aa9b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:33.457244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca554e3e0604d619934ef9c00b3aa9b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:33.459095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:33.461894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:33.658207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca554e3e0604d619934ef9c00b3aa9b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:33.659563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca554e3e0604d619934ef9c00b3aa9b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:33.660179 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca554e3e0604d619934ef9c00b3aa9b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:33.667635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca554e3e0604d619934ef9c00b3aa9b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:33.680688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca554e3e0604d619934ef9c00b3aa9b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:33.733081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1228/2474] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:33 2022] DELETE /identity/v3/users/3ca554e3e0604d619934ef9c00b3aa9b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:33.737759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:33.738233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:33.738338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:33.756085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:33.757330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:33.816355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1247/2475] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:33.826134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:33.827557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:33.884924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26f185-a70f-4f05-8181-a6268103e1d7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:33.888670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:33.889348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:33.889634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:33.889915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:33.899454 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:33.900877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a86a38-b4a7-48e3-b832-ce16b516992b service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:33.910564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1229/2476] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:34.362003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:34.367341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:34.380470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3d926-aaed-4cb2-ac15-1e6ae3e7b411 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:34.388749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:34.388749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.388749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.388749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] PATH_INFO: `/v3/projects/a285734c876649f89a753f86795e24c4` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.389914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a285734c876649f89a753f86795e24c4)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:34.391825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:34.504898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a285734c876649f89a753f86795e24c4', 'request_id': 'req-f05e089f-9f83-4ca3-9943-923c1bc7e46d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:34.510796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a285734c876649f89a753f86795e24c4', 'request_id': 'req-f05e089f-9f83-4ca3-9943-923c1bc7e46d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:34.589359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a285734c876649f89a753f86795e24c4', 'request_id': 'req-f05e089f-9f83-4ca3-9943-923c1bc7e46d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:34.602321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] The token cache is being invalidate because project a285734c876649f89a753f86795e24c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:34.602611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:34.603234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-f05e089f-9f83-4ca3-9943-923c1bc7e46d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:34.604812 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1248/2477] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:34 2022] DELETE /identity/v3/projects/a285734c876649f89a753f86795e24c4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:34.609751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.610233 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.610368 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.628475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:34.629707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:34.689765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1230/2478] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:34.697555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.697898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.698187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.717406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:34.718598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:34.727073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1249/2479] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:34.731061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:34.741177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff8844-991c-4ef4-8b19-68e09f7238b9 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:34.745828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:34.746475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.746754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.747056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.747776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:34.748085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:34.749715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:34.761141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1231/2480] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:34.765333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:34.775062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb88447-df0f-4ba9-bba7-b1810f38078d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:34.779803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:34.780447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.780731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.781004 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.781733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:34.783630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:34.796921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:34.860710 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1250/2481] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:34 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:34.864874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:34.874855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a581261-c02b-4632-807a-97aaad614266 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:34.882798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:34.934182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:34.934852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:34.935133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:34.935409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:34.936118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:34.938204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:35.076830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1232/2482] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:34 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:35.080867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:35.091687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e78ff2-88b2-43c9-83fb-3ba3a47229ae admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:35.096334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:35.097000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:35.097380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:35.097758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:35.098397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:35.100011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:35.110808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1251/2483] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:35.115616 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:35.126017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d2c2-d5c1-47a9-ac6c-0e760b05b083 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:35.132682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:35.133308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:35.133595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:35.133897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:35.134628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:35.136235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af832e6-7622-4257-9258-a858e645dcd8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:35.146902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1233/2484] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:35.150960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:35.162585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61af3ef-4c6b-4e1e-9c0c-8f62e23f010d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:35.167471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:35.168773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:35.169049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:35.169419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] PATH_INFO: `/v3/projects/3e5c8e1f3d2f402bafc06ee20c37270a/users/00e0e70823ab46439ad7007ff2d31dc3/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:35.187232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5c8e1f3d2f402bafc06ee20c37270a, user_id=00e0e70823ab46439ad7007ff2d31dc3, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:35.190140 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:35.264485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1252/2485] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:35 2022] PUT /identity/v3/projects/3e5c8e1f3d2f402bafc06ee20c37270a/users/00e0e70823ab46439ad7007ff2d31dc3/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:35.276273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:35.281508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:35.281508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:35.305403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] MFA Rules not processed for user `00e0e70823ab46439ad7007ff2d31dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:35.306635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:35.387840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1234/2486] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:35.399116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:35.401040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:35.466933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f8f16-ee26-4c60-89e8-89cd29769051 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:35.471308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:35.472024 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:35.472312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:35.472596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:35.514980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:35.514980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89444075-aa38-4fa0-81ab-1f46ed47bf23 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:35.514980 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1253/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:45.338433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:45.401533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cac5a-f106-4041-855e-cc22debe6da3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:45.406042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:45.406961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:45.407249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:45.407540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] PATH_INFO: `/v3/users/00e0e70823ab46439ad7007ff2d31dc3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:45.408778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e0e70823ab46439ad7007ff2d31dc3)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:45.410782 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:45.432971 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:45.488881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:45.611584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e0e70823ab46439ad7007ff2d31dc3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:45.612797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e0e70823ab46439ad7007ff2d31dc3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:45.613446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e0e70823ab46439ad7007ff2d31dc3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:45.620250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e0e70823ab46439ad7007ff2d31dc3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:45.633914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e0e70823ab46439ad7007ff2d31dc3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:45.715050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1235/2488] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:45 2022] DELETE /identity/v3/users/00e0e70823ab46439ad7007ff2d31dc3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:45.719555 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:45.720094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:45.720380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:45.738806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:45.740034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:45.799183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1254/2489] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:45.809796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:45.811234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:45.875237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5008-0195-48f7-95d8-dd861b5c782f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:45.883367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:45.883367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:45.883367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:45.883367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:45.891537 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:45.893053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098d8ca-fc29-415c-a28c-05588805f3aa service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:45.903391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1236/2490] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:46.347409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:46.353435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:46.363306 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663e47-0bc3-40c3-9c0e-52da790ac2d6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:46.368616 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:46.370079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.370438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.370722 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] PATH_INFO: `/v3/projects/3e5c8e1f3d2f402bafc06ee20c37270a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.371962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5c8e1f3d2f402bafc06ee20c37270a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:46.373966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:46.505386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5c8e1f3d2f402bafc06ee20c37270a', 'request_id': 'req-bb6eaf37-9eb4-49db-bdfc-0430f776e875'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:46.511014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5c8e1f3d2f402bafc06ee20c37270a', 'request_id': 'req-bb6eaf37-9eb4-49db-bdfc-0430f776e875'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:46.600018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5c8e1f3d2f402bafc06ee20c37270a', 'request_id': 'req-bb6eaf37-9eb4-49db-bdfc-0430f776e875'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:46.611892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] The token cache is being invalidate because project 3e5c8e1f3d2f402bafc06ee20c37270a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:46.612180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:46.612838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-bb6eaf37-9eb4-49db-bdfc-0430f776e875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:46.614395 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1255/2491] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:46 2022] DELETE /identity/v3/projects/3e5c8e1f3d2f402bafc06ee20c37270a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:46.619223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.619699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.619827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.638128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:46.639485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:46.699028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1237/2492] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:46.707302 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.707586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.707860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.726698 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:46.727888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:46.735875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1256/2493] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:46.740091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:46.750009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6673ae-b2b0-41d4-9a1d-20d8076a0af5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:46.755373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:46.756014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.756295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.756571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.757322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:46.757633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:46.759216 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:46.771121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1238/2494] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:46.775314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:46.784604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee780-ea6c-4f74-ae1a-ed983858ee0d None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:46.789369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:46.790019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.790303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.790576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.791266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:46.793144 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:46.871813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1257/2495] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:46 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:46.876093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:46.886777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d81ba-3b9c-43e8-8bc0-e00f9cc58242 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:46.912461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:46.942673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:46.943337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:46.943627 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:46.943911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:46.944613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:46.946594 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.072669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1239/2496] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:46 2022] POST /identity/v3/users => generated 282 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:47.076653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.087347 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2086a2f-01ae-40d0-b4f4-adaddd3f5ede admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.092480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.093399 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.093684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.093959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.094700 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.096322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.107073 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1258/2497] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:47.111092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.121757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1668-c6fe-4dfa-9332-fad19ed1e3af admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.126740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.127430 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.127716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.128072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.129352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.130940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bc23-f138-48f6-aad6-de1d2a2b9da7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.142913 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1240/2498] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:47.146938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.157920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fc322-8a54-416a-a801-897406d6b4ab admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.163281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.164097 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.167679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.167992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] PATH_INFO: `/v3/projects/e5d88a822d5e4be4be939d8f593ae9c5/users/1d1b42d1f44f4f1eb90a7ff627edc7ac/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.183773 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d88a822d5e4be4be939d8f593ae9c5, user_id=1d1b42d1f44f4f1eb90a7ff627edc7ac, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.186504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.263528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1259/2499] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:47 2022] PUT /identity/v3/projects/e5d88a822d5e4be4be939d8f593ae9c5/users/1d1b42d1f44f4f1eb90a7ff627edc7ac/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:47.273719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.274159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.274501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.302884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] MFA Rules not processed for user `1d1b42d1f44f4f1eb90a7ff627edc7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:47.304101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:47.368503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1241/2500] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:47.372857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.394413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:47.435526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce0b44-066f-4ba0-8354-4bdded2de56e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.440437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.447400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.447400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.447400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.447400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.447400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.598052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1260/2501] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:47 2022] POST /identity/v3/projects => generated 365 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:47.602308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.612849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e184348-1447-49bc-bb4b-65802e331102 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.667878 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.668555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.668820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.669097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.669826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.671748 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.756151 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1242/2502] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:47 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:47.759974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.771308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9e89-1abb-4fb2-bf85-35e7096a38b6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.776186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.776905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.777220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.777502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.778226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.779827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.791080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1261/2503] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:47.795318 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.805463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1187bb-f1e1-4dd4-a689-e85194766bd2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.810065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.810938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.811224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.811502 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] PATH_INFO: `/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d/users/926724eb2cc94fb69d8636804555ef1e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.826166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f85299e6b0648f1a305e0a4ccc9bb6d, user_id=926724eb2cc94fb69d8636804555ef1e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.828864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.900299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1243/2504] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:47 2022] PUT /identity/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d/users/926724eb2cc94fb69d8636804555ef1e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:47.904330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.915243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6b49f-ab00-4638-958c-c170bb510e12 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.976743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.977532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.977933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.978220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.978961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.980666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453617-7c8c-4fb8-9c88-8045d44ef0ae admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:47.991160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1262/2505] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:47.995365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.002313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:48.006989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048862b-32ec-4b53-85bd-d6053db02e50 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.011623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.012500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.012784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.013060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] PATH_INFO: `/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d/users/926724eb2cc94fb69d8636804555ef1e/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.015036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f85299e6b0648f1a305e0a4ccc9bb6d, user_id=926724eb2cc94fb69d8636804555ef1e, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.017797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.029945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1244/2506] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:47 2022] PUT /identity/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d/users/926724eb2cc94fb69d8636804555ef1e/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:48.041120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.041521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.041854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.069810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] MFA Rules not processed for user `926724eb2cc94fb69d8636804555ef1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:48.071080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.143234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1263/2507] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:48.155972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.157784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.241757 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f166e7-4617-45f3-8d77-077bc00752bb admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.245734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.246567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.247063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.247218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.263238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.263238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.274675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1245/2508] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:48.506643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.516508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba5a17-d2e2-468e-a829-83ed0a5d646f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.520641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.521355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.521644 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.521926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.535033 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:48.580095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.582120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.592360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1264/2509] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:50.917554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:50.919705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:50.982362 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44854482-fbcf-48ac-b601-01125ee1055a service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:50.986086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:50.986739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:50.987025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:50.987330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:50.989125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:51.053356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:51.054790 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff5164-ff96-490e-8dca-e5248638e25e service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:51.064312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1246/2510] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:52.167648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:52.259471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86127b37-376c-4cbf-b4dc-8305e3cdb882 service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:52.264259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:52.265139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:52.265452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:52.265732 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] PATH_INFO: `/v3/users/1d1b42d1f44f4f1eb90a7ff627edc7ac` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:52.266934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1b42d1f44f4f1eb90a7ff627edc7ac)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:52.268888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:52.440254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1b42d1f44f4f1eb90a7ff627edc7ac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:52.441145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1b42d1f44f4f1eb90a7ff627edc7ac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:52.441856 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1b42d1f44f4f1eb90a7ff627edc7ac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:52.449745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1b42d1f44f4f1eb90a7ff627edc7ac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:52.462286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1b42d1f44f4f1eb90a7ff627edc7ac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:52.532440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1265/2511] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:52 2022] DELETE /identity/v3/users/1d1b42d1f44f4f1eb90a7ff627edc7ac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:52.537417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:52.537930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:52.538019 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:52.555465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:52.556779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:52.595425 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:52.617093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1247/2512] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:52.628308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:52.630046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:52.665789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:52.687325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdcc20-ddaa-4bed-9ab9-b1a6c66832eb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:52.691061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:52.691701 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:52.691988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:52.692266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:52.701466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:52.703332 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:52.712739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1266/2513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:53.391336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:53.392871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:53.411714 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3e7eb-2a15-41b6-9fa7-c2d27acb6e81 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:53.416795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:53.418138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:53.418138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:53.418363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] PATH_INFO: `/v3/projects/e5d88a822d5e4be4be939d8f593ae9c5` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:53.419794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d88a822d5e4be4be939d8f593ae9c5)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:53.421977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:53.545795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d88a822d5e4be4be939d8f593ae9c5', 'request_id': 'req-7e415f25-574b-475d-b257-c5bf8b3a36b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.558637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d88a822d5e4be4be939d8f593ae9c5', 'request_id': 'req-7e415f25-574b-475d-b257-c5bf8b3a36b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.660834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d88a822d5e4be4be939d8f593ae9c5', 'request_id': 'req-7e415f25-574b-475d-b257-c5bf8b3a36b9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.674793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] The token cache is being invalidate because project e5d88a822d5e4be4be939d8f593ae9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:53.675467 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.676385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.678607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1248/2514] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:53 2022] DELETE /identity/v3/projects/e5d88a822d5e4be4be939d8f593ae9c5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:53.682416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:53.694533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e794-1c5f-481a-a250-77940026cc66 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:53.758676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:53.759509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:53.759863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:53.760461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] PATH_INFO: `/v3/users/926724eb2cc94fb69d8636804555ef1e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:53.761548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926724eb2cc94fb69d8636804555ef1e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:53.764016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:53.939866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926724eb2cc94fb69d8636804555ef1e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.941255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926724eb2cc94fb69d8636804555ef1e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.941987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926724eb2cc94fb69d8636804555ef1e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.949815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926724eb2cc94fb69d8636804555ef1e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:53.962964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0fa67c20-ee7a-4677-8063-55770624ea65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926724eb2cc94fb69d8636804555ef1e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.030101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1267/2515] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:53 2022] DELETE /identity/v3/users/926724eb2cc94fb69d8636804555ef1e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:54.502277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:54.504877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:54.520460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e415f25-574b-475d-b257-c5bf8b3a36b9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:54.582040 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:54.583083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:54.583591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:54.583870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] PATH_INFO: `/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:54.585087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f85299e6b0648f1a305e0a4ccc9bb6d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:54.587071 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:54.705374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f85299e6b0648f1a305e0a4ccc9bb6d', 'request_id': 'req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.711282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f85299e6b0648f1a305e0a4ccc9bb6d', 'request_id': 'req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.767722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f85299e6b0648f1a305e0a4ccc9bb6d', 'request_id': 'req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.770524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:54.780098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] The token cache is being invalidate because project 1f85299e6b0648f1a305e0a4ccc9bb6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:54.780384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.781011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3c5c1005-c423-44e6-94e3-72ddd54e4e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:54.782584 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1249/2516] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:54 2022] DELETE /identity/v3/projects/1f85299e6b0648f1a305e0a4ccc9bb6d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:54.787539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:54.788013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:54.788133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:54.806354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:54.807553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:54.859037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:54.870276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1268/2517] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:54.878749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:54.879038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:54.879320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:54.897144 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:54.898370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:54.906338 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1250/2518] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:54.910829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:54.920408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218c2d9-a831-4d69-9159-931ab6a39599 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:54.925104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:54.925775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:54.926061 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:54.926337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:54.927164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:54.927470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:54.929052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:54.940888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1269/2519] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:54.945000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:54.954524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e5520-e4a6-4b00-a6aa-4c7f33f3dd7d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:54.959474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:54.960062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:54.960351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:54.960628 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:54.961349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:54.963243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.088098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1251/2520] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:54 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:55.092454 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.103469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96f55-8499-4a1f-a644-41aa0fb4f1f2 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.164829 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.165537 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.165822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.166106 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.166820 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.168774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.269800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1270/2521] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:55 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:55.273819 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.284686 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae912ffc-ac70-47aa-9d59-a93f335e6f0e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.289433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.290087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.290403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.290678 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.291397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.292987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.303128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1252/2522] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:55.310313 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.320447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819bab9-cde0-4c14-9f6e-f2762c39c145 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.325129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.325815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.326100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.326411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.327137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.328728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533c1a6-e016-478b-8b36-01995fa2d4af admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.339823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1271/2523] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:55.343669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.354161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c31309-689a-486b-8081-72860bfaa639 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.358812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.359676 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.359960 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.360239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] PATH_INFO: `/v3/projects/92deb50600f84d2592f32d0a4ce39095/users/77daca5ac5814cb28f66d4f3eebb726f/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.374428 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92deb50600f84d2592f32d0a4ce39095, user_id=77daca5ac5814cb28f66d4f3eebb726f, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.377397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.494321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1253/2524] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:55 2022] PUT /identity/v3/projects/92deb50600f84d2592f32d0a4ce39095/users/77daca5ac5814cb28f66d4f3eebb726f/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:55.505499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.505792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.506067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.533689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] MFA Rules not processed for user `77daca5ac5814cb28f66d4f3eebb726f`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:55.534825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:55.597844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1272/2525] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:55.602386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.663358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289100c-6a39-4d77-ab43-bbee108aa2f8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.668191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.668967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.669300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.669640 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.670288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.672186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.770825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1254/2526] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:55 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:55.775087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.788318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21db174-1ab9-46e9-b67e-42db9a566673 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.839479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:55.849428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.852356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.852356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.852356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.852356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.853619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.922926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1273/2527] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:55 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:55.927556 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.938695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49e25-5422-406a-a3f5-26ac885147a2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.944334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.945003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.946308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.946308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.946623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.948107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.959193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1255/2528] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:45:55.963221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:55.974910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53242404-b846-4bb1-aaed-167e2a4432b5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:55.978216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:55.978879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:55.979194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:55.979473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:55.980231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:55.982012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9b3dc-a8a1-4b0f-9643-b19c5464afe5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:55.992766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1274/2529] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:55.996825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.007746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b96be-0f0c-4ac3-9baa-63b2ae0e16bf admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.012480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.013384 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.013698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.013976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] PATH_INFO: `/v3/projects/4e05d7a45aae450cac04dd0126aab1f8/users/967aff31d2e34ef38ddda52544a00073/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.030417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e05d7a45aae450cac04dd0126aab1f8, user_id=967aff31d2e34ef38ddda52544a00073, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.033147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.116358 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1256/2530] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:55 2022] PUT /identity/v3/projects/4e05d7a45aae450cac04dd0126aab1f8/users/967aff31d2e34ef38ddda52544a00073/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:56.128113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.128409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.128749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.155998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] MFA Rules not processed for user `967aff31d2e34ef38ddda52544a00073`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:56.157255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:56.222580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1275/2531] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:56.227121 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.274242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:56.291078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e7ac-6d0e-4764-b65b-37efd6ab1910 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.295828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.296487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.296771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.297045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.297827 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.299684 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.390220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1257/2532] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:56 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:56.394381 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.405726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022f53b-2442-49ee-b5cc-aa3182966235 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.463576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.464256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.464542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.464821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.465622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.467532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.562158 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1276/2533] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:56 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:56.566793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.577399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589d8cf-f409-4317-9ef9-ee9f8652cca5 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.582248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.582768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.583054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.583331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.584093 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.585740 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.596196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1258/2534] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:56.600111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.610307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472d213-804d-4f94-819e-69051fd4d4c4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.615495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.616370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.616654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.616932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] PATH_INFO: `/v3/projects/b0018ad6fc32408081fccc0bacb63116/users/a351078556134487ab2ec37f60e3385a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.632179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0018ad6fc32408081fccc0bacb63116, user_id=a351078556134487ab2ec37f60e3385a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.634862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.699113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1277/2535] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:56 2022] PUT /identity/v3/projects/b0018ad6fc32408081fccc0bacb63116/users/a351078556134487ab2ec37f60e3385a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:56.703110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.712914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4eba-a3ae-45de-aef9-ef2f3ee93bc6 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.767679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.768334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.768618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.768896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.769681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.771261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09f8a-acca-4294-bf57-3cd9ce78ff79 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.780690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1259/2536] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:56.784737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.794877 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd98c-c624-4163-8415-9071dc7de0fe admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:56.799406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:56.800308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.801094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.801403 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] PATH_INFO: `/v3/projects/b0018ad6fc32408081fccc0bacb63116/users/a351078556134487ab2ec37f60e3385a/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.803374 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0018ad6fc32408081fccc0bacb63116, user_id=a351078556134487ab2ec37f60e3385a, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:56.806034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:56.816002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1278/2537] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:56 2022] PUT /identity/v3/projects/b0018ad6fc32408081fccc0bacb63116/users/a351078556134487ab2ec37f60e3385a/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:56.826194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:56.826486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:56.826763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:56.853441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] MFA Rules not processed for user `a351078556134487ab2ec37f60e3385a`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:56.854662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:56.924694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1260/2538] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:45:56.935731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:56.937452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:56.951421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:56.998647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d8064-b40f-456e-a51d-5b26810daa50 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:57.002659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:57.003309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:57.003593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:57.003871 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:57.014184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:57.015646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de6a5-bda2-436d-a782-e4f9b3815a78 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:57.025731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1279/2539] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:57.157775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:57.163683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:57.181545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd08989-a44f-472a-8bde-240020cabb1d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:57.186961 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:57.187681 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:57.188072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:57.188397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:57.221374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:57.244924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:57.246389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:57.255843 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1261/2540] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:45:57.514509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:57.514894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:57.515191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:57.532703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:57.533947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:57.589774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1280/2541] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:57.612287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:57.613773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:57.687508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d55da6-2536-49fc-bb05-a965468ca0f1 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:57.693034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:57.693731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:57.694099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:57.694587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:57.703235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:57.705014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757fce4-cc37-4c72-bdfa-e0c3efb158f3 service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:57.713614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1262/2542] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:45:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:46:01.480133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:01.556281 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2809603-b19b-48cd-9f5b-f449280df690 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:01.561665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:01.563495 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:01.564000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:01.564482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] PATH_INFO: `/v3/users/77daca5ac5814cb28f66d4f3eebb726f` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:01.566349 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77daca5ac5814cb28f66d4f3eebb726f)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:01.570960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:01.763574 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77daca5ac5814cb28f66d4f3eebb726f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:01.764436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77daca5ac5814cb28f66d4f3eebb726f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:01.765492 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77daca5ac5814cb28f66d4f3eebb726f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:01.782569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77daca5ac5814cb28f66d4f3eebb726f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:01.782569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:01.789861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77daca5ac5814cb28f66d4f3eebb726f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:01.860197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1281/2543] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:46:01 2022] DELETE /identity/v3/users/77daca5ac5814cb28f66d4f3eebb726f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:01.868834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:01.868834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:01.868834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:01.899126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:01.899126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:01.984285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1263/2544] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:46:01.992005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:01.994385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:02.087505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd414a-8804-41fb-a33e-6f1573914789 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:02.091151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:02.091754 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:02.092210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:02.092432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:02.110953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:02.112401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:02.123457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1282/2545] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:02.624553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:02.624553 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:02.633390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421e73-510a-4941-8ab7-3b95fe59c715 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:02.647134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:02.648165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:02.648583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:02.649044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] PATH_INFO: `/v3/projects/92deb50600f84d2592f32d0a4ce39095` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:02.650634 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92deb50600f84d2592f32d0a4ce39095)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:02.653438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:02.748509 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92deb50600f84d2592f32d0a4ce39095', 'request_id': 'req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:02.759953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92deb50600f84d2592f32d0a4ce39095', 'request_id': 'req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:02.763910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:02.825471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92deb50600f84d2592f32d0a4ce39095', 'request_id': 'req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:02.843698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] The token cache is being invalidate because project 92deb50600f84d2592f32d0a4ce39095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:46:02.844183 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:02.844970 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:02.851508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1264/2546] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:46:02 2022] DELETE /identity/v3/projects/92deb50600f84d2592f32d0a4ce39095 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:02.855319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:02.866280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a2d1-0b17-4d9f-b477-6688d48e3844 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:02.945382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:02.947071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:02.947460 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:02.947842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] PATH_INFO: `/v3/users/967aff31d2e34ef38ddda52544a00073` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:02.949353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=967aff31d2e34ef38ddda52544a00073)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:02.951385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:03.180118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967aff31d2e34ef38ddda52544a00073'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:03.180770 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967aff31d2e34ef38ddda52544a00073'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:03.181453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967aff31d2e34ef38ddda52544a00073'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:03.192183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967aff31d2e34ef38ddda52544a00073'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:03.208625 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967aff31d2e34ef38ddda52544a00073'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:03.301822 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:03.309110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1283/2547] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:46:02 2022] DELETE /identity/v3/users/967aff31d2e34ef38ddda52544a00073 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:46:03.837572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:03.840519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:03.855920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bda4c-1bd6-47e6-8624-7babf7f7d85a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:03.938600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:03.939516 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:03.939803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:03.940075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] PATH_INFO: `/v3/projects/4e05d7a45aae450cac04dd0126aab1f8` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:03.941867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e05d7a45aae450cac04dd0126aab1f8)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:03.944204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:04.136743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e05d7a45aae450cac04dd0126aab1f8', 'request_id': 'req-c6acf711-1ea4-4f07-b296-8520b72a70fe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.149402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e05d7a45aae450cac04dd0126aab1f8', 'request_id': 'req-c6acf711-1ea4-4f07-b296-8520b72a70fe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.208606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e05d7a45aae450cac04dd0126aab1f8', 'request_id': 'req-c6acf711-1ea4-4f07-b296-8520b72a70fe'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.222450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] The token cache is being invalidate because project 4e05d7a45aae450cac04dd0126aab1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:46:04.223089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.224184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.226287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1265/2548] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:46:03 2022] DELETE /identity/v3/projects/4e05d7a45aae450cac04dd0126aab1f8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:46:04.230308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:04.231779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:04.252212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853fc3e-0eed-4d9a-aa6c-eccf3433b523 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:04.318937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:04.319914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:04.320239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:04.320528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] PATH_INFO: `/v3/users/a351078556134487ab2ec37f60e3385a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:04.321801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a351078556134487ab2ec37f60e3385a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:04.323832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:04.685985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a351078556134487ab2ec37f60e3385a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.686636 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a351078556134487ab2ec37f60e3385a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.687234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a351078556134487ab2ec37f60e3385a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.709356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a351078556134487ab2ec37f60e3385a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.709356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-cf8cedbf-1539-42bd-8bb3-d47f3f02144e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a351078556134487ab2ec37f60e3385a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:04.801801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1284/2549] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:46:04 2022] DELETE /identity/v3/users/a351078556134487ab2ec37f60e3385a => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:05.253024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:05.254666 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:05.277011 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf711-1ea4-4f07-b296-8520b72a70fe admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:05.376746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:05.377954 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:05.378359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:05.378731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] PATH_INFO: `/v3/projects/b0018ad6fc32408081fccc0bacb63116` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:05.380054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0018ad6fc32408081fccc0bacb63116)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:05.382323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:05.415588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:05.528868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0018ad6fc32408081fccc0bacb63116', 'request_id': 'req-20f29890-2eb9-4e8a-984b-05418ce0b5b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:05.536458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0018ad6fc32408081fccc0bacb63116', 'request_id': 'req-20f29890-2eb9-4e8a-984b-05418ce0b5b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:05.635057 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0018ad6fc32408081fccc0bacb63116', 'request_id': 'req-20f29890-2eb9-4e8a-984b-05418ce0b5b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:05.649058 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] The token cache is being invalidate because project b0018ad6fc32408081fccc0bacb63116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:46:05.649480 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:05.650796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-20f29890-2eb9-4e8a-984b-05418ce0b5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:05.652747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1266/2550] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:46:05 2022] DELETE /identity/v3/projects/b0018ad6fc32408081fccc0bacb63116 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:05.659608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:05.660711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:05.661057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:05.680422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:05.681962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:05.757984 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1285/2551] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:05.767210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:05.767452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:05.767739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:05.792710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:05.794137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:05.802685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1267/2552] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:05.806917 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:05.817954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8197f9-cef7-4adc-a744-a852840446b6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:05.823032 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:05.824243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:05.824529 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:05.824807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:05.825591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:46:05.825900 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:05.827684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:05.844862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1286/2553] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:46:05.850111 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:05.861192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce4335-23b8-428d-a121-a30ec18a0ead None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:05.866620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:05.867486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:05.867768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:05.868043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:05.868853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:05.870828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.023163 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1268/2554] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:46:05 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:46:06.024604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:06.043821 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857c4dd-145e-424a-b1de-d05e13b1cfee admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:06.047597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:06.125320 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:06.126693 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.127551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.128134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.129295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:06.131966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.227113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1287/2555] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:46:06 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:46:06.231374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:06.247223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975533e3-f584-49f0-9d1c-5139e5233a32 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:06.254299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:06.256247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.256857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.257940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.261239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:06.261239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.274287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1269/2556] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:46:06.278390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:06.291558 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e8a1-2f0c-4492-b878-b520c5059aab admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:06.297124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:06.297940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.298296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.298646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.299457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:06.301910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c2e98-b6c0-4a2f-a918-e08fcd9cee15 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.317295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1288/2557] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:46:06.318915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:06.329564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c182c-5d72-42fa-b8ae-ba99c70494a4 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:06.337615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:06.338608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.338969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.339319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] PATH_INFO: `/v3/projects/5a7cf32af35a4543870f52d6abd0bd6e/users/be9cedcf8871499aa4bf9559de21f23d/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.355598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7cf32af35a4543870f52d6abd0bd6e, user_id=be9cedcf8871499aa4bf9559de21f23d, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:06.358549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.449759 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1270/2558] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:46:06 2022] PUT /identity/v3/projects/5a7cf32af35a4543870f52d6abd0bd6e/users/be9cedcf8871499aa4bf9559de21f23d/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:46:06.461482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcf87-a4f0-497e-a954-fdd213e5f078 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.461805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcf87-a4f0-497e-a954-fdd213e5f078 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.462094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcf87-a4f0-497e-a954-fdd213e5f078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.497986 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-549fcf87-a4f0-497e-a954-fdd213e5f078 None None] MFA Rules not processed for user `be9cedcf8871499aa4bf9559de21f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:06.499230 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-549fcf87-a4f0-497e-a954-fdd213e5f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:06.582169 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1289/2559] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:06.591696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:06.593657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:06.672692 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129e60-2b8d-41a1-8acd-aac814e422a4 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:06.679797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:06.680635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:06.681024 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:06.681436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:06.696793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:06.698710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:06.716363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1271/2560] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:16.574711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:16.704797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:18.841858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:18.842376 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:18.842484 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:18.860437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:18.861682 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:18.918789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1290/2561] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Jul 12 14:46:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:18.931172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:18.932565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:19.001847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ed34-e00c-4752-a801-a0694b8386cb service glance] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:19.006841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:19.007514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:19.007794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:19.008067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:19.016076 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:19.017668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c335c2-ac91-449c-9824-94cb0bd8cd1b service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:19.026247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1272/2562] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:46:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:46:28.911035 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=25 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:29.022900 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=32 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:29.237225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:29.300565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f9c1-20e1-4e90-b5c4-2dcabc989427 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:29.305723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:29.306590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:29.306869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:29.307142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] PATH_INFO: `/v3/users/be9cedcf8871499aa4bf9559de21f23d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:29.308333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9cedcf8871499aa4bf9559de21f23d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:29.310294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:29.497221 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9cedcf8871499aa4bf9559de21f23d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:29.497881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9cedcf8871499aa4bf9559de21f23d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:29.498612 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9cedcf8871499aa4bf9559de21f23d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:29.505989 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9cedcf8871499aa4bf9559de21f23d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:29.518613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9cedcf8871499aa4bf9559de21f23d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:29.592113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1291/2563] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:46:29 2022] DELETE /identity/v3/users/be9cedcf8871499aa4bf9559de21f23d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:29.596266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:29.596558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:29.596833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:29.615448 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:29.620372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:29.680205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1273/2564] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:29.689739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:29.691132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:29.747152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ecfdc-09dd-4bad-a2a2-ea86343af9dc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:29.750880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:29.751766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:29.752056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:29.752334 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:29.761601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:29.763011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d134f3f-7a6a-4a44-92a7-46f73216a825 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:29.773002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1292/2565] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:30.167929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.169441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:30.181536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24339e-1d2c-4931-bf92-40205604976d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.187150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.188192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.188477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.188749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] PATH_INFO: `/v3/projects/5a7cf32af35a4543870f52d6abd0bd6e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.189979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7cf32af35a4543870f52d6abd0bd6e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.193350 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.323286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7cf32af35a4543870f52d6abd0bd6e', 'request_id': 'req-09be303d-e103-47b0-afd0-cd493502e058'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:30.329038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7cf32af35a4543870f52d6abd0bd6e', 'request_id': 'req-09be303d-e103-47b0-afd0-cd493502e058'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:30.394653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7cf32af35a4543870f52d6abd0bd6e', 'request_id': 'req-09be303d-e103-47b0-afd0-cd493502e058'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:30.406579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] The token cache is being invalidate because project 5a7cf32af35a4543870f52d6abd0bd6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:46:30.406871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:30.407545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-09be303d-e103-47b0-afd0-cd493502e058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:46:30.409063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1274/2566] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:46:30 2022] DELETE /identity/v3/projects/5a7cf32af35a4543870f52d6abd0bd6e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:46:30.414116 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.414655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.414655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.433196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:30.434406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:30.455551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:30.501213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1293/2567] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:30.506645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.510694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.510694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.526164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:30.527485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:30.535836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1275/2568] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:30.542468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.552704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90e1db-7af2-470e-a3de-c9158b6a43cd None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.557333 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.557977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.558263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.558538 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.559265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:46:30.559565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.561173 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.577493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1294/2569] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:46:30.578478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.587976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb72b6-458e-48b2-91ad-4492fa9f4983 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.592641 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.593331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.593621 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.593903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.595195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.597220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.706774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1276/2570] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:46:30 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:46:30.712521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.723309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbc306-2bf5-444c-8f04-e8a6b3af0c9d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.778159 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.778816 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.779099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.779370 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.780047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.781976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.857746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1295/2571] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:46:30 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:46:30.861906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.872865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb4f59-f0ca-4bde-8f2e-78895383d0e9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.877864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.878531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.878873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.879091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.879822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.881455 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.895665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1277/2572] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:46:30.897269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.907752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4e330-9bae-4aea-ae48-44d2074c894c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.913163 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.913836 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.914127 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.915913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.915913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.916776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2e43-e2cf-41f2-9bea-99f963090dfe admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.927763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1296/2573] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:46:30.931922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.941771 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fc086-fc48-42d1-8b30-f9b2c148ddfa admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.946562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.947435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.947725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.948002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] PATH_INFO: `/v3/projects/8ddd54b29e2645b3a99d5a92db287bb0/users/8d2cf38a5efc4417ac0b44a58eec56cd/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.962687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ddd54b29e2645b3a99d5a92db287bb0, user_id=8d2cf38a5efc4417ac0b44a58eec56cd, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.965398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:31.048785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1278/2574] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:46:30 2022] PUT /identity/v3/projects/8ddd54b29e2645b3a99d5a92db287bb0/users/8d2cf38a5efc4417ac0b44a58eec56cd/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:46:31.117255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353ac8-7260-4279-a45b-8ae8df80853a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:31.117553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353ac8-7260-4279-a45b-8ae8df80853a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:31.117840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353ac8-7260-4279-a45b-8ae8df80853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:31.145280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-1e353ac8-7260-4279-a45b-8ae8df80853a None None] MFA Rules not processed for user `8d2cf38a5efc4417ac0b44a58eec56cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:31.146499 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-1e353ac8-7260-4279-a45b-8ae8df80853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:31.210855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1297/2575] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:31.222270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:31.224727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:31.246520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:31.283528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7629-adf7-45f8-87d1-1c5aaa780ab7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:31.287876 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:31.288554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:31.288842 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:31.289123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:31.297656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:31.299086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:31.308436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1279/2576] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:46:36.360138 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df9f2-70ef-42bd-86c0-815f1a76cc87 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:36.360643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df9f2-70ef-42bd-86c0-815f1a76cc87 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:36.360740 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8df9f2-70ef-42bd-86c0-815f1a76cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:36.382351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-be8df9f2-70ef-42bd-86c0-815f1a76cc87 None None] MFA Rules not processed for user `eceac3a84f1d4eef8fecde82551fb0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:36.384355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-be8df9f2-70ef-42bd-86c0-815f1a76cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:36.400194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:36.459368 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1298/2577] 10.222.0.30 () {58 vars in 967 bytes} [Tue Jul 12 14:46:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:36.474508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:36.477747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:36.582069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8ce43-f571-4793-8338-f80156107af4 service glance] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:36.587285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59997304919b432184f842b10ab9bba8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:36.588198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:36.588519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:36.588742 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:36.597511 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:36.599895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:36.608505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1280/2578] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Jul 12 14:46:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:46:38.299876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:38.300350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:38.300445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:38.319668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] MFA Rules not processed for user `917bfc7c68384536a6945af83308f789`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:38.320882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:38.396059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1299/2579] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:38.714892 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:38.716361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:38.780564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda869de-6337-4768-87df-1c6dd57af53b service swift] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:38.784632 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:38.785542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:38.785898 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:38.786246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:38.788683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:38.799772 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:38.801834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:38.811527 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1281/2580] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:48.388189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=42 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:48.807979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=75 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:07.925037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:07.928025 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:07.940783 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35e6e-2abe-439d-8d32-e66ecd77458c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:07.944759 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:07.945507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:07.945985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:07.946263 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:07.948438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:08.025635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:08.027364 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4c065-3bac-4d02-928e-a08faa9b42e1 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:08.052546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1300/2581] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:11.034249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:11.046036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659990f0-cd1d-4cde-aa24-a4bf645cd157 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:11.051001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:11.051897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:11.052182 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:11.052499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] PATH_INFO: `/v3/users/8d2cf38a5efc4417ac0b44a58eec56cd` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:11.053904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2cf38a5efc4417ac0b44a58eec56cd)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:11.056541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:11.226940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2cf38a5efc4417ac0b44a58eec56cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:11.227618 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2cf38a5efc4417ac0b44a58eec56cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:11.228291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2cf38a5efc4417ac0b44a58eec56cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:11.238983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2cf38a5efc4417ac0b44a58eec56cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:11.256943 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2cf38a5efc4417ac0b44a58eec56cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:11.336164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1282/2582] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:11 2022] DELETE /identity/v3/users/8d2cf38a5efc4417ac0b44a58eec56cd => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:11.341756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:11.342240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:11.342325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:11.361074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:11.364412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:11.441425 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1301/2583] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:11.448773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:11.451008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:11.517573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc4c7f-b9f1-4b8d-a0b7-3d03f8c9a994 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:11.521719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:11.522475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:11.522834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:11.523187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:11.534888 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:11.536484 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:11.548369 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1283/2584] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:12.129690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.131162 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:12.145665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c50c9-6035-493c-b6c6-20ce4e7d82dd None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.151087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.152122 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.152387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.152743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] PATH_INFO: `/v3/projects/8ddd54b29e2645b3a99d5a92db287bb0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.153978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ddd54b29e2645b3a99d5a92db287bb0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.155952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.230113 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.231590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:12.245077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3ed16-9e44-4d7f-a972-41a12cb24371 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.249671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.250536 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.250952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.251234 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] PATH_INFO: `/v3/users/8f118c0ba8494e6e9f5a4488ebf8d220` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.252683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f118c0ba8494e6e9f5a4488ebf8d220)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.254650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.293791 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ddd54b29e2645b3a99d5a92db287bb0', 'request_id': 'req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.299372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ddd54b29e2645b3a99d5a92db287bb0', 'request_id': 'req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.379618 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ddd54b29e2645b3a99d5a92db287bb0', 'request_id': 'req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.391716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] The token cache is being invalidate because project 8ddd54b29e2645b3a99d5a92db287bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:12.392007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.392721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-00e1ea4a-46b9-4f19-a6eb-6255d0b3715a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.394518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1302/2585] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:12 2022] DELETE /identity/v3/projects/8ddd54b29e2645b3a99d5a92db287bb0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:12.400001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756d02-ac51-4a71-93fe-4f1142395dc0 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.400286 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756d02-ac51-4a71-93fe-4f1142395dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.400560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756d02-ac51-4a71-93fe-4f1142395dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.418279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-89756d02-ac51-4a71-93fe-4f1142395dc0 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:12.419474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-89756d02-ac51-4a71-93fe-4f1142395dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:12.431264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:12.466243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f118c0ba8494e6e9f5a4488ebf8d220'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.466944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f118c0ba8494e6e9f5a4488ebf8d220'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.467548 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f118c0ba8494e6e9f5a4488ebf8d220'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.474620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f118c0ba8494e6e9f5a4488ebf8d220'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.480437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1303/2586] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:47:12.488141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.490229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.490229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.490431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:12.491940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f118c0ba8494e6e9f5a4488ebf8d220'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.506851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:12.508889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:12.517663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1304/2587] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:12.521638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.530961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64972d-2378-4633-9aca-f905e8df5525 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.535584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.536200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.539591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.539591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.539591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:47:12.539591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.539591 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.550516 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1305/2588] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:47:12.554518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.560901 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1284/2589] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:12 2022] DELETE /identity/v3/users/8f118c0ba8494e6e9f5a4488ebf8d220 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:12.567208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbf450-4d30-4859-a91e-b846cc0a9f53 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.632711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.634007 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.634283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.634554 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.635357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.637279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.721107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1306/2590] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:12 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:47:12.722397 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.732492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950028-382e-485b-8912-dc6be057ebd8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.788447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.789110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.789426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.789707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.790395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.792300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.803705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.806372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:12.819220 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57306e-6528-494e-9909-882800f2a2cf admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.825047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.825948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.826233 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.826513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] PATH_INFO: `/v3/projects/52569bf4eba24c2a9fa0ef8c26b8b22d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.827764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52569bf4eba24c2a9fa0ef8c26b8b22d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.829703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.863529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1285/2591] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:12 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:12.867934 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.877735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b97b0-c9f0-494c-a6fc-4b9b91cb2718 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.882394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.883583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.883870 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.884150 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.884865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.886540 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.896433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1286/2592] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:47:12.900211 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.909483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62fe90-7ade-400f-abb8-df20f48013dc admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.914597 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:12.915224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:12.915504 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:12.915785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:12.916483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:12.918055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:12.928287 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1287/2593] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:47:12.933631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:12.943541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52569bf4eba24c2a9fa0ef8c26b8b22d', 'request_id': 'req-efaa7616-f722-4795-adc8-710fbac5f607'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:12.944417 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7588-a96c-4045-8a9d-12838b5779a7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:12.949421 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52569bf4eba24c2a9fa0ef8c26b8b22d', 'request_id': 'req-efaa7616-f722-4795-adc8-710fbac5f607'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:13.002230 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.003477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.003764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.004045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] PATH_INFO: `/v3/projects/e89c4c765f5243eaa314231429d20598/users/2b123a78349c4daba0502a4e9e98245f/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.018301 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89c4c765f5243eaa314231429d20598, user_id=2b123a78349c4daba0502a4e9e98245f, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.020937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab7e0a-6419-41e2-bbc8-610f1ece202f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.025449 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52569bf4eba24c2a9fa0ef8c26b8b22d', 'request_id': 'req-efaa7616-f722-4795-adc8-710fbac5f607'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:13.038624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] The token cache is being invalidate because project 52569bf4eba24c2a9fa0ef8c26b8b22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:13.038914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:13.040113 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-efaa7616-f722-4795-adc8-710fbac5f607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:13.041738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1307/2594] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:12 2022] DELETE /identity/v3/projects/52569bf4eba24c2a9fa0ef8c26b8b22d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:13.046394 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee58de-306f-478f-b051-130695d602ae None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.046662 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee58de-306f-478f-b051-130695d602ae None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.046943 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee58de-306f-478f-b051-130695d602ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.059287 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:13.065262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e0ee58de-306f-478f-b051-130695d602ae None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:13.066463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e0ee58de-306f-478f-b051-130695d602ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.074391 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1308/2595] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:47:13.085895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.086186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.086464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.104182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:13.113222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.113361 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1288/2596] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:12 2022] PUT /identity/v3/projects/e89c4c765f5243eaa314231429d20598/users/2b123a78349c4daba0502a4e9e98245f/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:13.116893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.117257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.118115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.134896 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:13.136094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.165118 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1309/2597] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:47:13.169992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:13.174155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.183750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03976f-4abb-4bbb-8cdc-7ee5525cf73e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.188729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.189443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.189729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.190042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.190769 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:47:13.191079 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.192666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.200514 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1289/2598] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:47:13.208351 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1310/2599] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:47:13.209708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.211333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.212506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.224199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453af3e6-3fd5-4ff5-bd84-080456121dec admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.229691 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.233835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.233835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.233835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.233835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.233835 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.270648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1290c-1315-41e1-bc09-f84985b26a56 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.274880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.275528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.275813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.276107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.285773 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.291709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.300391 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1290/2600] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:13.346862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1311/2601] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:47:13.351043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.361507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c65a-5950-4283-908d-ed5f410ded46 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.419331 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.419986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.420311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.420528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.421236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.423176 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.500701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1291/2602] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:13.505198 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.516190 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb34e-ac39-4b84-8d1f-49353210d30f admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.520888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.521568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.521851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.522125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.522850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.524461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.534293 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1312/2603] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:13.538187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.548044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374356-d2c2-41ad-ad52-c387d57058c1 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.552637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.553398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.553680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.553955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.555195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.556774 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd93f9d-ff44-40b9-8f90-016fa17bfcdf admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.566840 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1292/2604] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:47:13.571365 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.581392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b3f6-70f3-4673-ad20-80f0ba77b361 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.586089 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.586970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.587282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.587561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] PATH_INFO: `/v3/projects/35438fec2e37438e99201870e0d650af/users/f926d674e2dc4eb58d7fe6379b79310b/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.602834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35438fec2e37438e99201870e0d650af, user_id=f926d674e2dc4eb58d7fe6379b79310b, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.605509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.685132 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1313/2605] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:13 2022] PUT /identity/v3/projects/35438fec2e37438e99201870e0d650af/users/f926d674e2dc4eb58d7fe6379b79310b/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:13.690165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59da23-016c-4551-b4a1-4786bd39d0f6 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.690458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59da23-016c-4551-b4a1-4786bd39d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.690749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59da23-016c-4551-b4a1-4786bd39d0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.710073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-ab59da23-016c-4551-b4a1-4786bd39d0f6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:13.718240 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ab59da23-016c-4551-b4a1-4786bd39d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.760765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:13.768924 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1293/2606] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:13.778758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:13.780587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:13.842439 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ecf6-3609-4714-86d4-e27f453e2573 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:13.846312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:13.846962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:13.847246 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:13.847523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:13.857774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:13.859348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:13.872248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1314/2607] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:14.585738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df2ab-da32-4ed0-a5c2-7a5ea9e713e4 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:14.586781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df2ab-da32-4ed0-a5c2-7a5ea9e713e4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:14.587286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df2ab-da32-4ed0-a5c2-7a5ea9e713e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:14.616957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-135df2ab-da32-4ed0-a5c2-7a5ea9e713e4 None None] MFA Rules not processed for user `2b123a78349c4daba0502a4e9e98245f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:14.618381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-135df2ab-da32-4ed0-a5c2-7a5ea9e713e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:14.706443 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1294/2608] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:14.717868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:14.719331 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:14.755270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:14.778615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71277db5-3de9-4bdd-b0ce-0203555e3637 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:14.782266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:14.782910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:14.783194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:14.783471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:14.791670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:14.793093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcaf4a-d3c3-43db-afe3-a236f839b293 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:14.801819 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1315/2609] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:16.546969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:16.547487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:16.547599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:16.566326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] MFA Rules not processed for user `2b123a78349c4daba0502a4e9e98245f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:16.567526 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:16.575269 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1295/2610] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:19.630422 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249707-b4ec-4183-bddc-c2fe8ac5a4a5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:19.631416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249707-b4ec-4183-bddc-c2fe8ac5a4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:19.631841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6249707-b4ec-4183-bddc-c2fe8ac5a4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:19.673854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f6249707-b4ec-4183-bddc-c2fe8ac5a4a5 None None] MFA Rules not processed for user `f926d674e2dc4eb58d7fe6379b79310b`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:19.675265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f6249707-b4ec-4183-bddc-c2fe8ac5a4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:19.779735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1316/2611] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:19.794886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:19.796779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:19.952277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75020d7-b65e-40cd-94f8-4c06d212894c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:19.961944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:19.963086 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:19.963605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:19.964127 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:19.971950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:19.976219 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:19.978629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:19.993367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1296/2612] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:22.493125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:22.493595 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:22.493869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:22.513283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] MFA Rules not processed for user `f926d674e2dc4eb58d7fe6379b79310b`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:22.514486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:22.521442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1317/2613] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:30.042276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:30.579957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:30.590643 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d1b4-ab88-45e2-8853-d8a2863ecb8c service nova] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:30.594567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:30.595242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:30.595702 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:30.595985 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:30.597804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:30.636804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:30.638286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:30.658446 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1297/2614] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:32.507739 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=58 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:34.530401 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:34.531973 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:34.641766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9bbe0-6a6e-41ff-8ece-83aa818361d5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:34.647713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:34.648480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:34.649452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:34.649806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:34.653456 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:34.670065 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:34.671546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:34.671680 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:34.674008 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:34.697524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882575b8-cba6-455d-9b43-8b4ecf208547 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:34.697800 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1318/2615] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:34.704947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:34.704947 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:34.705763 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:34.706201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] PATH_INFO: `/v3/users/a760d437b597441fa9a4595995c5bc6f` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:34.709221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a760d437b597441fa9a4595995c5bc6f)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:34.712272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:34.861868 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a760d437b597441fa9a4595995c5bc6f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:34.862523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a760d437b597441fa9a4595995c5bc6f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:34.863132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a760d437b597441fa9a4595995c5bc6f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:34.870066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a760d437b597441fa9a4595995c5bc6f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:34.882005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-79beffb5-d465-44b2-94da-aceb6d554825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a760d437b597441fa9a4595995c5bc6f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:34.956784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1298/2616] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:34 2022] DELETE /identity/v3/users/a760d437b597441fa9a4595995c5bc6f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:35.238704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:35.252618 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db33e96-9be8-4aae-af49-ddb4597604e8 service placement] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:35.346806 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:35.347679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.347958 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.348247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] PATH_INFO: `/v3/projects/3f550f8d11a74b8da5ad84607b79ac37` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.349594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f550f8d11a74b8da5ad84607b79ac37)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:35.351559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:35.466530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f550f8d11a74b8da5ad84607b79ac37', 'request_id': 'req-af667d5e-2d1f-4f4d-a357-6ec380e6155f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:35.471906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f550f8d11a74b8da5ad84607b79ac37', 'request_id': 'req-af667d5e-2d1f-4f4d-a357-6ec380e6155f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:35.516142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f550f8d11a74b8da5ad84607b79ac37', 'request_id': 'req-af667d5e-2d1f-4f4d-a357-6ec380e6155f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:35.524649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:35.528699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] The token cache is being invalidate because project 3f550f8d11a74b8da5ad84607b79ac37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:35.528999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:35.529699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-af667d5e-2d1f-4f4d-a357-6ec380e6155f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:35.531249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1319/2617] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:35 2022] DELETE /identity/v3/projects/3f550f8d11a74b8da5ad84607b79ac37 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:35.536300 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.536590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.536894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.592645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:35.593937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:35.646153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:35.654409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1299/2618] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:35.662297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.662826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.663290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.683115 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:35.684342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:35.692453 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1320/2619] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:35.698973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:35.709073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d5929-1d5c-459f-9705-a54291d22a03 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:35.713844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:35.714507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.714793 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.715075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.715805 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:47:35.716118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:35.717796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:35.729733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1300/2620] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:47:35.734139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:35.744053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704300f2-a4e9-4f8d-a055-4b3ed28bf61a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:35.748826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:35.749506 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.749793 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.750071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.750762 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:35.753026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:35.830992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1321/2621] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:35 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:35.843945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:35.854953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4cfd6-7786-4b5d-9f2d-a40ec0c8dc4d admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:35.910349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:35.911006 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:35.911288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:35.911588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:35.912271 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:35.914252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.006280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1301/2622] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:35 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:36.011152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.023275 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102ec0-708e-4098-b6b3-4790562811c4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.028297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.029014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.029343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.029680 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.030468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.032173 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.044213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1322/2623] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:36.048250 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.059500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f93b1-6576-4087-afee-ff6a713671fc admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.064346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.065012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.065332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.065638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.066375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.068089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef48d9a-447c-45b3-8314-982c0bb41c07 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.078984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1302/2624] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:36.087687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.098271 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c33e7-4433-4cfb-922e-a6a3ffe312fb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.103661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.104545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.104870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.105167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] PATH_INFO: `/v3/projects/2619bed2e1d04e7c9e8ba6aa85e94b75/users/d77ea9de01bb47a88b2033f8ae1e813e/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.122432 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2619bed2e1d04e7c9e8ba6aa85e94b75, user_id=d77ea9de01bb47a88b2033f8ae1e813e, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.125206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.202305 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1323/2625] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:36 2022] PUT /identity/v3/projects/2619bed2e1d04e7c9e8ba6aa85e94b75/users/d77ea9de01bb47a88b2033f8ae1e813e/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:36.214420 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.214715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.215004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.242202 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] MFA Rules not processed for user `d77ea9de01bb47a88b2033f8ae1e813e`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:36.243427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:36.330052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1303/2626] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:36.334539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.401655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec4b6b-914c-471a-b507-0d4ab82840eb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.406503 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.407208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.409216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.409216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.409216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.410488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.496167 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1324/2627] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:36 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:36.499745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.511403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc842346-664b-4db5-9a0c-5947e9bde509 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.566214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:36.575342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.577818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.577818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.577818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.577818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.579226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.688829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1304/2628] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:36 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:36.693072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.708419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61d0b9-d75d-4304-902e-a4eca3d7437d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.714151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.714814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.715444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.715444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.716182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.717827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.729992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1325/2629] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:36.734249 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.745148 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437a505-a7ed-4424-9d0a-28200a99d3c2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.749891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.750707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.751045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.751314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] PATH_INFO: `/v3/projects/cd96ead61174421bb9f9ff55839369f3/users/024199db2e9141e3968c5187ff5eaecc/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.768459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd96ead61174421bb9f9ff55839369f3, user_id=024199db2e9141e3968c5187ff5eaecc, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.771227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.854730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1305/2630] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:36 2022] PUT /identity/v3/projects/cd96ead61174421bb9f9ff55839369f3/users/024199db2e9141e3968c5187ff5eaecc/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:36.858775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.870686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39a63e-6c74-4081-a747-f8dfad38ec1b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.931860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.932576 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.932861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.933136 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.933926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.935507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07215d32-b563-4837-8e7d-a466e3ce23da admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.945936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1326/2631] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:36.950160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:36.960525 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607dcd81-9054-485e-b5f9-03c8104191cd admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:36.965311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:36.966189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:36.966474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:36.966756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] PATH_INFO: `/v3/projects/cd96ead61174421bb9f9ff55839369f3/users/024199db2e9141e3968c5187ff5eaecc/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:36.968724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd96ead61174421bb9f9ff55839369f3, user_id=024199db2e9141e3968c5187ff5eaecc, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:36.971583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:36.984323 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1306/2632] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:36 2022] PUT /identity/v3/projects/cd96ead61174421bb9f9ff55839369f3/users/024199db2e9141e3968c5187ff5eaecc/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:37.001337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:37.001572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:37.001854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:37.014815 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:37.029736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] MFA Rules not processed for user `024199db2e9141e3968c5187ff5eaecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:37.031178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:37.103704 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1327/2633] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:37.115353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:37.117066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:37.175780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd29-997b-46ba-a966-520799312731 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:37.179700 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:37.180887 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:37.181195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:37.181560 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:37.191532 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:37.192978 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2151a24-902e-48f5-be26-b7386b7d2d4e service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:37.202661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1307/2634] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:37.231463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:37.294731 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879181-bc54-4a1a-b093-dc99d4024a5b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:37.299385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:37.300262 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:37.300548 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:37.300825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] PATH_INFO: `/v3/users/d77ea9de01bb47a88b2033f8ae1e813e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:37.302077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77ea9de01bb47a88b2033f8ae1e813e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:37.304010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:37.501475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77ea9de01bb47a88b2033f8ae1e813e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:37.502229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77ea9de01bb47a88b2033f8ae1e813e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:37.503226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77ea9de01bb47a88b2033f8ae1e813e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:37.512868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77ea9de01bb47a88b2033f8ae1e813e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:37.524879 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77ea9de01bb47a88b2033f8ae1e813e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:37.585551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1328/2635] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:37 2022] DELETE /identity/v3/users/d77ea9de01bb47a88b2033f8ae1e813e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:37.594018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:37.594383 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:37.594850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:37.614134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:37.615498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:37.675533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1308/2636] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:37.685912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:37.687400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:37.701183 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:37.770855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcc366-51d6-40b7-b281-5618798e9825 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:37.774881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:37.775590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:37.775875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:37.776152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:37.786977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:37.788580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:37.801593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1329/2637] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:38.264078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:38.265750 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:38.279944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e91d-e746-4d18-895b-063cb4875517 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:38.284877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:38.285789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:38.286069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:38.286344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] PATH_INFO: `/v3/projects/2619bed2e1d04e7c9e8ba6aa85e94b75` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:38.287565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2619bed2e1d04e7c9e8ba6aa85e94b75)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:38.290146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:38.415143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:38.418779 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2619bed2e1d04e7c9e8ba6aa85e94b75', 'request_id': 'req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.424227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2619bed2e1d04e7c9e8ba6aa85e94b75', 'request_id': 'req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.488332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2619bed2e1d04e7c9e8ba6aa85e94b75', 'request_id': 'req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.511642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] The token cache is being invalidate because project 2619bed2e1d04e7c9e8ba6aa85e94b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:38.512458 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.513665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.515487 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1309/2638] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:38 2022] DELETE /identity/v3/projects/2619bed2e1d04e7c9e8ba6aa85e94b75 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:38.523589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:38.529946 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696aa23-89ab-4bcd-9d4d-873a775eeddd service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:38.594006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:38.594905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:38.595193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:38.595475 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] PATH_INFO: `/v3/users/024199db2e9141e3968c5187ff5eaecc` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:38.596690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] RBAC: Authorizing `identity:delete_user(user_id=024199db2e9141e3968c5187ff5eaecc)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:38.598656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:38.767266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024199db2e9141e3968c5187ff5eaecc'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.768048 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024199db2e9141e3968c5187ff5eaecc'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.768566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024199db2e9141e3968c5187ff5eaecc'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.776747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024199db2e9141e3968c5187ff5eaecc'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.790474 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-81e88b4d-9351-493f-9704-6337f68b521b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024199db2e9141e3968c5187ff5eaecc'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:38.860970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1330/2639] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:38 2022] DELETE /identity/v3/users/024199db2e9141e3968c5187ff5eaecc => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:39.294836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:39.296308 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:39.317035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e38c5-f7cd-420e-8ef3-0272520aca2c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:39.383940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:39.384816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.385102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.385410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] PATH_INFO: `/v3/projects/cd96ead61174421bb9f9ff55839369f3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.386638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd96ead61174421bb9f9ff55839369f3)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:39.388589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:39.514236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd96ead61174421bb9f9ff55839369f3', 'request_id': 'req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:39.522196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd96ead61174421bb9f9ff55839369f3', 'request_id': 'req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:39.594976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd96ead61174421bb9f9ff55839369f3', 'request_id': 'req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:39.609894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] The token cache is being invalidate because project cd96ead61174421bb9f9ff55839369f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:39.610213 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:39.610895 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-c0b89552-c1e4-4a4c-b149-f3c83cde6be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:39.612648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1310/2640] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:39 2022] DELETE /identity/v3/projects/cd96ead61174421bb9f9ff55839369f3 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:39.617699 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.617995 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.618282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.637134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:39.638382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:39.650148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:39.713670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1331/2641] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:39.721385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.721685 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.721987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.741883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:39.743134 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:39.751106 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1311/2642] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:39.755894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:39.766208 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9bcfa-42c1-40b7-89b4-d2b232808faf None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:39.770913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:39.771578 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.771864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.772145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.772926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:47:39.773267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:39.775424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:39.787049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1332/2643] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:47:39.791983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:39.801247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14bff-548e-4eda-ae95-1b4a857fd9a8 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:39.806710 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:39.807486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.807696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.808009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.808669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:39.810623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:39.908242 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1312/2644] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:39 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:39.912304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:39.923250 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a7ce1-d3c8-4503-b847-0ed70fbed41b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:39.979839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:39.980508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:39.980797 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:39.981075 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:39.981808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:39.983721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.111666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1333/2645] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:39 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:40.116075 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.127462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643a40-b0a0-430b-b362-d9d8dfaff184 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.132284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.133110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.133426 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.133705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.134435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.136302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.147110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1313/2646] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:40.151443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.173677 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda99-748d-4094-ad3b-8002b1351ee9 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.177805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.178479 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.178766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.179047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.180400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.183346 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba52124-7c39-4f7d-be7e-93bbfa90d49b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.192864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1334/2647] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:40.197399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.209969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f41f0-3471-47ac-b877-53eb92bbe72d admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.215405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.216741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.217147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.217573 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] PATH_INFO: `/v3/projects/27803da3bb7d4341a4fceeb695c477bd/users/d886f62c4c1e4c76b11855750fb4f4b5/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.237650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27803da3bb7d4341a4fceeb695c477bd, user_id=d886f62c4c1e4c76b11855750fb4f4b5, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.241137 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.307304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:40.312754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1314/2648] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:40 2022] PUT /identity/v3/projects/27803da3bb7d4341a4fceeb695c477bd/users/d886f62c4c1e4c76b11855750fb4f4b5/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:40.324307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.324598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.324875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.355239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] MFA Rules not processed for user `d886f62c4c1e4c76b11855750fb4f4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:40.356457 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:40.422330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1335/2649] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:40.426708 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.490486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08ac58-5208-4de4-bd59-699bff451e8d admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.495728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.496387 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.496675 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.496950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.497695 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.499647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.607950 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1315/2650] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:40 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:40.611863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.623601 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81548f3e-b252-48f3-a0b4-619b349b0cfc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.630266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:40.686279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.686960 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.687269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.687545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.688247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.690245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.769071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1336/2651] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:40 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:40.773285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.783857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56f4b6-a1e2-490f-a69d-1440647d9d0d admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.788488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.789716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.790126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.790521 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.791407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.793004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.804229 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1316/2652] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:40.809641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.820406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faf1cf-6107-468e-a2ea-5f33b7bd4423 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.825280 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.825952 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.826245 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.826524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.827253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.828853 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ef20d-e436-42a7-bef6-004bc5027c9c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.839081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1337/2653] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:47:40.843351 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:40.853855 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77e0a-d48c-43ba-8bd9-93382d96051c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:40.858503 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:40.859400 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.859679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.859948 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] PATH_INFO: `/v3/projects/449fe4d9c21e4ebfb0cf9762f79c3a73/users/82f71288669841a9a4e651b8bd951a3e/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:40.876280 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449fe4d9c21e4ebfb0cf9762f79c3a73, user_id=82f71288669841a9a4e651b8bd951a3e, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:40.879246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:40.974828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1317/2654] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:40 2022] PUT /identity/v3/projects/449fe4d9c21e4ebfb0cf9762f79c3a73/users/82f71288669841a9a4e651b8bd951a3e/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:40.985533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:40.985827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:40.986125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.015110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] MFA Rules not processed for user `82f71288669841a9a4e651b8bd951a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:41.016329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:41.082046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1338/2655] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:41.087042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.150305 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95425039-d660-40e1-b16e-c4ea122cf393 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.155056 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.155724 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.156012 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.156291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.156995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.158935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.256160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1318/2656] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:41 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:41.260692 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.272186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14ee2-9bd6-47be-9d8e-221119fdfa2b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.328654 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.329375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.329667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.329949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.330687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.332613 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.397240 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1339/2657] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:41 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:41.401460 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.412313 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196b9ef-3288-450b-b778-e8a563fb9283 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.417646 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.418307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.418595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.418874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.419602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.421227 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.434193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1319/2658] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:41.438400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.449114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea37ba-2ee4-41ae-8592-6705d71833c8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.453814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.454752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.455042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.455321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] PATH_INFO: `/v3/projects/52343e7550dc45e1b1467391bae22bba/users/503f4d2b48cf4c5eb5269da4173fbd3f/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.457673 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:41.470725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52343e7550dc45e1b1467391bae22bba, user_id=503f4d2b48cf4c5eb5269da4173fbd3f, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.473405 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.555840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1340/2659] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:41 2022] PUT /identity/v3/projects/52343e7550dc45e1b1467391bae22bba/users/503f4d2b48cf4c5eb5269da4173fbd3f/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:41.559837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.569966 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350f90d-1568-44b6-bf0f-9a8965ccc9ff admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.590210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:41.630248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.630957 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.631245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.631526 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.632264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.633911 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2625-6c32-4f2a-99b9-284042b173c5 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.644690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1320/2660] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:41.649764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.662873 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb3c67-06c1-4b66-b752-5fb2709c12cc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.668131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.669041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.669355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.669631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] PATH_INFO: `/v3/projects/52343e7550dc45e1b1467391bae22bba/users/503f4d2b48cf4c5eb5269da4173fbd3f/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.671713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52343e7550dc45e1b1467391bae22bba, user_id=503f4d2b48cf4c5eb5269da4173fbd3f, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.674384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.685817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1341/2661] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:41 2022] PUT /identity/v3/projects/52343e7550dc45e1b1467391bae22bba/users/503f4d2b48cf4c5eb5269da4173fbd3f/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:41.696096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.696386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.696668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.724955 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] MFA Rules not processed for user `503f4d2b48cf4c5eb5269da4173fbd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:41.726245 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:41.795816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1321/2662] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:41.807020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:41.808679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:41.895360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023bad-202f-49e5-a196-44c1c425a0e4 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:41.899270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:41.899921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:41.900204 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:41.900477 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:41.909766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:41.911185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:41.920526 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1342/2663] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:43.011307 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:43.012715 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:43.026336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24be1b-71a1-4f05-80e7-a7b279e2e5ba None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:43.030235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:43.030890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:43.031169 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:43.031441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:43.090833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:43.092275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:43.102188 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1322/2664] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:47.975938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:47.984635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab793c90-99cf-4c5d-9b18-6e44a93287ec service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:47.988253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:47.988904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:47.989200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:47.989462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:48.046994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:48.048497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2fec2-e0ba-45db-8601-7ba88e07e059 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:48.058068 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1343/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:52.610342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:52.658533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:52.676456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d8636-988b-4405-ac2a-f423d4430011 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:52.679465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:52.680320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:52.680648 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:52.680863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] PATH_INFO: `/v3/users/d886f62c4c1e4c76b11855750fb4f4b5` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:52.682125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d886f62c4c1e4c76b11855750fb4f4b5)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:52.684034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:52.826850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd886f62c4c1e4c76b11855750fb4f4b5'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:52.827510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd886f62c4c1e4c76b11855750fb4f4b5'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:52.828167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd886f62c4c1e4c76b11855750fb4f4b5'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:52.835264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd886f62c4c1e4c76b11855750fb4f4b5'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:52.847867 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd886f62c4c1e4c76b11855750fb4f4b5'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:52.898937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1323/2666] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:52 2022] DELETE /identity/v3/users/d886f62c4c1e4c76b11855750fb4f4b5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:52.904094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:52.904587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:52.904690 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:52.921792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:52.923001 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:52.968330 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:52.982353 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1344/2667] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:52.992032 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:52.993495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:53.051520 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6744ab-fb4d-428d-8528-2d7a32bdad02 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:53.055277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:53.055928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:53.056784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:53.057069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:53.066295 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:53.067728 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:53.081221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1324/2668] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:47:53.491120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:53.491120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:53.502411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e28f8-9b8a-4b05-b54d-eb507ffd0cbc None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:53.507451 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:53.508325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:53.508607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:53.508882 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] PATH_INFO: `/v3/projects/27803da3bb7d4341a4fceeb695c477bd` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:53.510267 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] RBAC: Authorizing `identity:delete_project(project_id=27803da3bb7d4341a4fceeb695c477bd)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:53.515609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:53.637344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27803da3bb7d4341a4fceeb695c477bd', 'request_id': 'req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.642848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27803da3bb7d4341a4fceeb695c477bd', 'request_id': 'req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.721940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27803da3bb7d4341a4fceeb695c477bd', 'request_id': 'req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.734141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] The token cache is being invalidate because project 27803da3bb7d4341a4fceeb695c477bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:53.734433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.735066 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.736607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1345/2669] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:53 2022] DELETE /identity/v3/projects/27803da3bb7d4341a4fceeb695c477bd => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:53.740741 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:53.750761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2f11-1da4-4e8a-bde0-b9fdbdceb85e service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:53.806210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:53.807203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:53.807483 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:53.807756 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] PATH_INFO: `/v3/users/82f71288669841a9a4e651b8bd951a3e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:53.808926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f71288669841a9a4e651b8bd951a3e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:53.810884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:53.816149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:53.973381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f71288669841a9a4e651b8bd951a3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.974052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f71288669841a9a4e651b8bd951a3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.974669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f71288669841a9a4e651b8bd951a3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.982022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f71288669841a9a4e651b8bd951a3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:53.994471 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f71288669841a9a4e651b8bd951a3e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.096990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1325/2670] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:53 2022] DELETE /identity/v3/users/82f71288669841a9a4e651b8bd951a3e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:54.523684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:54.525593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:54.539947 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd39c-2f14-49d1-9fd8-38647a19d88d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:54.598694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:54.599546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:54.599827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:54.600103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] PATH_INFO: `/v3/projects/449fe4d9c21e4ebfb0cf9762f79c3a73` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:54.601315 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449fe4d9c21e4ebfb0cf9762f79c3a73)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:54.603228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:54.703386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449fe4d9c21e4ebfb0cf9762f79c3a73', 'request_id': 'req-0d5815e0-c43f-433c-a2a9-16261ea52595'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.708927 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449fe4d9c21e4ebfb0cf9762f79c3a73', 'request_id': 'req-0d5815e0-c43f-433c-a2a9-16261ea52595'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.791207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449fe4d9c21e4ebfb0cf9762f79c3a73', 'request_id': 'req-0d5815e0-c43f-433c-a2a9-16261ea52595'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.802715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] The token cache is being invalidate because project 449fe4d9c21e4ebfb0cf9762f79c3a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:54.803049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.803686 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:54.805291 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1346/2671] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:54 2022] DELETE /identity/v3/projects/449fe4d9c21e4ebfb0cf9762f79c3a73 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:54.809749 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:54.811382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:54.824767 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee54ce3-118d-45c6-acf7-b5d7f240237a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:54.880133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:54.881046 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:54.881353 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:54.881629 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] PATH_INFO: `/v3/users/503f4d2b48cf4c5eb5269da4173fbd3f` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:54.882871 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503f4d2b48cf4c5eb5269da4173fbd3f)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:54.884818 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:55.153816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503f4d2b48cf4c5eb5269da4173fbd3f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.154394 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503f4d2b48cf4c5eb5269da4173fbd3f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.154999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503f4d2b48cf4c5eb5269da4173fbd3f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.161993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503f4d2b48cf4c5eb5269da4173fbd3f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.175824 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-931b95e1-1309-4129-a228-8e12a2a20750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503f4d2b48cf4c5eb5269da4173fbd3f'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.288298 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1326/2672] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:54 2022] DELETE /identity/v3/users/503f4d2b48cf4c5eb5269da4173fbd3f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:55.706766 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:55.708356 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:55.718063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:55.722249 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5815e0-c43f-433c-a2a9-16261ea52595 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:55.779985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:55.780845 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:55.781134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:55.781435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] PATH_INFO: `/v3/projects/52343e7550dc45e1b1467391bae22bba` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:55.782611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52343e7550dc45e1b1467391bae22bba)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:55.784513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:55.883350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52343e7550dc45e1b1467391bae22bba', 'request_id': 'req-eef8def1-39f2-41bd-be76-b79cdc16f093'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.888949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52343e7550dc45e1b1467391bae22bba', 'request_id': 'req-eef8def1-39f2-41bd-be76-b79cdc16f093'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.963012 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52343e7550dc45e1b1467391bae22bba', 'request_id': 'req-eef8def1-39f2-41bd-be76-b79cdc16f093'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.975568 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] The token cache is being invalidate because project 52343e7550dc45e1b1467391bae22bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:55.975857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.976561 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eef8def1-39f2-41bd-be76-b79cdc16f093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:55.978921 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1347/2673] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:55 2022] DELETE /identity/v3/projects/52343e7550dc45e1b1467391bae22bba => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:55.984972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:55.985493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:55.985595 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:55.993193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:56.004195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:56.005635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:56.063385 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1327/2674] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:56.071730 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.072030 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.072308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.091696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:56.092902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:56.101344 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1348/2675] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:56.106135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.115665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406da79-1a49-4394-bd47-bd018f9200ec None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.120304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.121054 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.121592 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.121937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.122803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:47:56.123491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.125135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.136802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1328/2676] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:47:56.141083 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.150909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199231a-3d0b-476b-9880-ecb84616723a None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.155611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.156265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.156767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.156914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.157532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.159424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.260206 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1349/2677] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:56.264347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.274677 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3141f-e824-48d2-a1e9-c188c912c450 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.332341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.333008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.333330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.333613 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.334312 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.336231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.446125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1329/2678] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:56.450642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.460858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a407-4e40-4cca-9945-413481f29d16 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.466252 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.466903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.467186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.467462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.468185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.470018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.480254 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1350/2679] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:56.484087 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.494160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498cec-bcf6-4f06-870e-8683f37dd9f8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.498638 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.499297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.499585 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.499863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.500582 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.502232 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac98231-d200-4737-bcc5-4c61c74c2448 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.512858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1330/2680] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:47:56.516663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.526860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593ceb-241b-4b0e-ae6e-4ee31ef12458 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.531497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.532360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.532639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.532909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] PATH_INFO: `/v3/projects/bf6090fa86b24bb98624ad8cdc8a6c2a/users/7f89d5c8f05c48678f9fb73c778496f8/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.547257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6090fa86b24bb98624ad8cdc8a6c2a, user_id=7f89d5c8f05c48678f9fb73c778496f8, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.549910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.645790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1351/2681] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:56 2022] PUT /identity/v3/projects/bf6090fa86b24bb98624ad8cdc8a6c2a/users/7f89d5c8f05c48678f9fb73c778496f8/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:56.657796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.658094 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.658404 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.684184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] MFA Rules not processed for user `7f89d5c8f05c48678f9fb73c778496f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:56.685418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:56.745392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:56.748890 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1331/2682] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:56.753753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.786934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:56.814847 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096099a9-3bcc-4c06-9bd0-0539899ac91e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.819667 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.820323 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.820611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.820916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.821676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.824092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:56.905831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1352/2683] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/projects => generated 389 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:56.910072 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:56.920929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b172-504a-40d9-bd02-86479912cf10 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:56.977528 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:56.978195 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:56.978477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:56.978754 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:56.979433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:56.981378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.053807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1332/2684] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:47:56 2022] POST /identity/v3/users => generated 301 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:47:57.058345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.069046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05abdb8-cecf-46a9-9793-81cdb97481b2 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.074317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.074965 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.075248 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.075524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.076253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.077867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.088196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1353/2685] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:57.092217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.102178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadccd-de7f-4e51-a34d-d83e24fb3bac admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.108136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.109286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.109570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.109841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] PATH_INFO: `/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a/users/0077322f13b6420bbc37bf3caa4fddf3/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.124800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fb0a85148c4f10b8a8eaf75dd9d72a, user_id=0077322f13b6420bbc37bf3caa4fddf3, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.127452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.178830 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1333/2686] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:57 2022] PUT /identity/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a/users/0077322f13b6420bbc37bf3caa4fddf3/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:57.183357 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.193372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dfe1a-be8b-4bec-85fc-74d2bc3efcd6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.249799 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.250466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.250750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.251027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.251774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.253640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d49c53-8b50-451a-86f7-5413f0b1cb67 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.263239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1354/2687] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:47:57.267339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.277474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252e4d-ad37-41f7-b357-aa583094454c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.282110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.282986 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.283270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.283549 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] PATH_INFO: `/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a/users/0077322f13b6420bbc37bf3caa4fddf3/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.285563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fb0a85148c4f10b8a8eaf75dd9d72a, user_id=0077322f13b6420bbc37bf3caa4fddf3, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.288236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.299531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1334/2688] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:47:57 2022] PUT /identity/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a/users/0077322f13b6420bbc37bf3caa4fddf3/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:57.311166 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.311466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.311747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.338200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] MFA Rules not processed for user `0077322f13b6420bbc37bf3caa4fddf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:47:57.339404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:57.406916 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1355/2689] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:47:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:47:57.418237 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.420008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:57.476642 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b486c-d0fa-4cb4-9aaf-0fba492cd03c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.480829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.481508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.481788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.482063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.490873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.492439 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.501777 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1335/2690] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:47:57.547013 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:57.548419 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:47:57.560666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d85525-3be4-4819-81a8-c5e7ecda9df8 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:57.564142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:57.564758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:57.565038 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:57.565341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:57.591175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:57.622108 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:57.623547 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef5ec4-d4d1-4fb7-a92f-a27c0012a17d service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:57.633464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1356/2691] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:02.359206 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:02.415463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:02.420826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6d1f2-96dc-4a80-b109-a2fe34cf72a2 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:02.425418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:02.429541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:02.429541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:02.429541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] PATH_INFO: `/v3/users/7f89d5c8f05c48678f9fb73c778496f8` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:02.429541 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f89d5c8f05c48678f9fb73c778496f8)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:02.430189 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:02.604241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f89d5c8f05c48678f9fb73c778496f8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:02.604851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f89d5c8f05c48678f9fb73c778496f8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:02.605513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f89d5c8f05c48678f9fb73c778496f8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:02.612880 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f89d5c8f05c48678f9fb73c778496f8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:02.625283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f89d5c8f05c48678f9fb73c778496f8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:02.711194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1336/2692] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:02 2022] DELETE /identity/v3/users/7f89d5c8f05c48678f9fb73c778496f8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:48:02.716358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:02.716867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:02.716975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:02.733895 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:02.735105 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:02.799458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1357/2693] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:02.807789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:02.809314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:02.870272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a4992-e19c-4f4b-a0fb-79eefdec7496 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:02.873858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:02.874515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:02.874963 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:02.875231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:02.886319 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:02.887894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:02.899569 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1337/2694] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:03.335736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:03.337239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:03.349047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ec7d7-30d5-4422-89d2-ee312a5b360e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:03.354814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:03.355674 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:03.355957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:03.356237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] PATH_INFO: `/v3/projects/bf6090fa86b24bb98624ad8cdc8a6c2a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:03.357467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6090fa86b24bb98624ad8cdc8a6c2a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:03.359386 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:03.483964 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6090fa86b24bb98624ad8cdc8a6c2a', 'request_id': 'req-6ae99521-cac6-4495-92fe-46cef4bea888'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.489494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6090fa86b24bb98624ad8cdc8a6c2a', 'request_id': 'req-6ae99521-cac6-4495-92fe-46cef4bea888'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.565833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6090fa86b24bb98624ad8cdc8a6c2a', 'request_id': 'req-6ae99521-cac6-4495-92fe-46cef4bea888'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.578423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] The token cache is being invalidate because project bf6090fa86b24bb98624ad8cdc8a6c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:03.578711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.579377 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.580971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1358/2695] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:03 2022] DELETE /identity/v3/projects/bf6090fa86b24bb98624ad8cdc8a6c2a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:03.585095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:03.595477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954797-0e40-4e07-a34b-47ecb69b6d45 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:03.652574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:03.653481 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:03.653766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:03.654043 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] PATH_INFO: `/v3/users/0077322f13b6420bbc37bf3caa4fddf3` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:03.655270 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0077322f13b6420bbc37bf3caa4fddf3)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:03.657221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:03.678673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:03.816444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0077322f13b6420bbc37bf3caa4fddf3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.817118 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0077322f13b6420bbc37bf3caa4fddf3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.817752 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0077322f13b6420bbc37bf3caa4fddf3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.824817 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0077322f13b6420bbc37bf3caa4fddf3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.836689 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8f769734-7c49-45d3-be05-4924e2f346cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0077322f13b6420bbc37bf3caa4fddf3'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:03.898721 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1338/2696] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:03 2022] DELETE /identity/v3/users/0077322f13b6420bbc37bf3caa4fddf3 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:04.375283 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:04.376742 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:04.391101 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae99521-cac6-4495-92fe-46cef4bea888 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:04.436776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:04.448738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:04.449643 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:04.449930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:04.450210 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] PATH_INFO: `/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:04.451392 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fb0a85148c4f10b8a8eaf75dd9d72a)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:04.453337 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:04.611464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fb0a85148c4f10b8a8eaf75dd9d72a', 'request_id': 'req-3755e1f8-ec35-464d-b96b-564ef6cedc97'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:04.616752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fb0a85148c4f10b8a8eaf75dd9d72a', 'request_id': 'req-3755e1f8-ec35-464d-b96b-564ef6cedc97'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:04.680410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fb0a85148c4f10b8a8eaf75dd9d72a', 'request_id': 'req-3755e1f8-ec35-464d-b96b-564ef6cedc97'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:04.693212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] The token cache is being invalidate because project 91fb0a85148c4f10b8a8eaf75dd9d72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:04.693521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:04.694155 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-3755e1f8-ec35-464d-b96b-564ef6cedc97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:04.695728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1359/2697] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:04 2022] DELETE /identity/v3/projects/91fb0a85148c4f10b8a8eaf75dd9d72a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:04.700788 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:04.701281 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:04.701399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:04.718683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:04.719886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:04.779096 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1339/2698] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:04.786842 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:04.787133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:04.787410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:04.805902 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:04.807664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:04.815586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1360/2699] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:04.819945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:04.829720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38869-c49e-4159-8205-07f80e86871f None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:04.834244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:04.834930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:04.835215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:04.835492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:04.836212 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:48:04.836519 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:04.838166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:04.849598 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1340/2700] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:48:04.854145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:04.863299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6103e1-4fae-4c04-bb33-0ef88b8b5ab5 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:04.867875 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:04.869109 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:04.869423 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:04.869703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:04.870415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:04.872314 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:04.990592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1361/2701] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:48:04 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:04.995266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.005522 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364e42a-951e-446b-9405-f9d25aa7d0fd admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.060981 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.061716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.061997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.062275 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.062959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.064858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.135352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1341/2702] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:48:04 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:05.136570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.147478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6fc40-d952-4ad2-ad6e-949a04cf62a5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.152542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.156438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.156438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.156438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.156438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.156438 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.167582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1362/2703] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:05.171529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.182424 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd76cc-e03a-4d14-8b92-da4d18eecfba admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.187185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.187936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.188222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.188500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.189263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.190852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd109f-db5a-45dd-a327-84120bdb977a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.202068 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1342/2704] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:05.206458 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.216830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd489693-a7ae-4bb3-aff7-50e27398f21d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.222189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.223100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.223511 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.223776 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] PATH_INFO: `/v3/projects/5dc03a15ae8b4dbf8b37e83eeedacd38/users/c12acb6e07174ad78cf6d6dacb48e28b/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.240242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc03a15ae8b4dbf8b37e83eeedacd38, user_id=c12acb6e07174ad78cf6d6dacb48e28b, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.243020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.323085 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1363/2705] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:05 2022] PUT /identity/v3/projects/5dc03a15ae8b4dbf8b37e83eeedacd38/users/c12acb6e07174ad78cf6d6dacb48e28b/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:05.333603 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.333899 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.334181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.340149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:05.361374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] MFA Rules not processed for user `c12acb6e07174ad78cf6d6dacb48e28b`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:05.362588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:05.426092 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1343/2706] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:05.430959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.492617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb61e-3dab-45df-8c48-126cde5e0ca8 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.497936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.498593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.498974 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.499259 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.499954 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.501903 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.574345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:05.580469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1364/2707] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:48:05 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:05.584558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.595663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c2591-f2fd-4739-981f-76304c007122 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.653141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.653839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.654126 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.654405 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.655122 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.657067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.742575 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1344/2708] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:48:05 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:05.747181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.757756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68ea82-8bb9-4fa0-828b-5fc87589af59 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.762959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.763611 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.763891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.764164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.764970 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.767011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.778006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1365/2709] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:05.781937 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.791912 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f29677-a184-4b8e-bef4-508ccf1e4d74 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.796578 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.797493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.797780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.798063 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] PATH_INFO: `/v3/projects/31c674ae326a499689fccfcb613fca46/users/2c6728ab3b1c4f1baf6909e453b6e58c/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.813228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c674ae326a499689fccfcb613fca46, user_id=2c6728ab3b1c4f1baf6909e453b6e58c, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.815879 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.886107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1345/2710] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:05 2022] PUT /identity/v3/projects/31c674ae326a499689fccfcb613fca46/users/2c6728ab3b1c4f1baf6909e453b6e58c/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:05.890020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.900137 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f7088-66fb-4858-b600-3b642476ebca admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.954994 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.955648 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.955929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.956205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.956919 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.958531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c5737-ae97-46ba-8bed-7e5ccd2e5a71 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:05.967956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1366/2711] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:05.972035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:05.982124 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64707c91-62a7-4b06-9121-6a7bf7edaf55 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:05.986743 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:05.987615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:05.987904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:05.988198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] PATH_INFO: `/v3/projects/31c674ae326a499689fccfcb613fca46/users/2c6728ab3b1c4f1baf6909e453b6e58c/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:05.990214 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c674ae326a499689fccfcb613fca46, user_id=2c6728ab3b1c4f1baf6909e453b6e58c, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:05.997310 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:06.086060 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1346/2712] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:05 2022] PUT /identity/v3/projects/31c674ae326a499689fccfcb613fca46/users/2c6728ab3b1c4f1baf6909e453b6e58c/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:06.097188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:06.098243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:06.098243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:06.124971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] MFA Rules not processed for user `2c6728ab3b1c4f1baf6909e453b6e58c`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:06.126273 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:06.193841 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1367/2713] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:06.205450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:06.207158 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:06.252463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:06.263007 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a03c-80aa-49bc-b8d4-e1cfb01e61dc admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:06.270991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:06.270991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:06.270991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:06.270991 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:06.277198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:06.278626 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:06.293317 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1347/2714] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:06.445890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:06.458713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d15369-2426-4719-9bcd-7d4f9dbc1234 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:06.463604 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:06.464247 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:06.464527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:06.464827 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:06.522496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:06.523854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beef796-dfd6-4d86-88fc-6de109edcf89 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:06.532673 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1368/2715] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:07.056631 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:07.117590 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd2a3c-d4ec-45a7-be05-576289872270 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:07.122832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:07.123694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:07.124031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:07.124261 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] PATH_INFO: `/v3/users/c12acb6e07174ad78cf6d6dacb48e28b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:07.125479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12acb6e07174ad78cf6d6dacb48e28b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:07.128623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:07.284545 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12acb6e07174ad78cf6d6dacb48e28b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:07.285231 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12acb6e07174ad78cf6d6dacb48e28b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:07.285835 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12acb6e07174ad78cf6d6dacb48e28b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:07.292673 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12acb6e07174ad78cf6d6dacb48e28b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:07.304775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12acb6e07174ad78cf6d6dacb48e28b'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:07.374178 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1348/2716] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:07 2022] DELETE /identity/v3/users/c12acb6e07174ad78cf6d6dacb48e28b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:07.379341 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:07.380594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:07.380594 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:07.397467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:07.398656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:07.402498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:07.457608 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1369/2717] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:07.467207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:07.468815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:07.527100 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7a4f-a41e-492d-b5e4-e998af574d75 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:07.530719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:07.531367 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:07.531650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:07.531932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:07.542602 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:07.544073 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:07.553906 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1349/2718] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:08.061466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:08.062940 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:08.075872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c052c0f-b1bb-493a-8332-8286d199edb7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:08.080606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:08.081437 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:08.081727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:08.082005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] PATH_INFO: `/v3/projects/5dc03a15ae8b4dbf8b37e83eeedacd38` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:08.083209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc03a15ae8b4dbf8b37e83eeedacd38)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:08.085200 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:08.219433 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc03a15ae8b4dbf8b37e83eeedacd38', 'request_id': 'req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.224926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc03a15ae8b4dbf8b37e83eeedacd38', 'request_id': 'req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.299214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc03a15ae8b4dbf8b37e83eeedacd38', 'request_id': 'req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.311174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] The token cache is being invalidate because project 5dc03a15ae8b4dbf8b37e83eeedacd38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:08.311407 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.312100 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.313681 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1370/2719] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:08 2022] DELETE /identity/v3/projects/5dc03a15ae8b4dbf8b37e83eeedacd38 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:08.317722 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:08.328203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f986ef-aabf-4d62-984a-6fed06c9d14e service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:08.333099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:08.387140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:08.388009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:08.388291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:08.388566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] PATH_INFO: `/v3/users/2c6728ab3b1c4f1baf6909e453b6e58c` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:08.389808 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6728ab3b1c4f1baf6909e453b6e58c)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:08.391707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:08.608255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6728ab3b1c4f1baf6909e453b6e58c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.608810 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6728ab3b1c4f1baf6909e453b6e58c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.609440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6728ab3b1c4f1baf6909e453b6e58c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.616663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6728ab3b1c4f1baf6909e453b6e58c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.636190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aae374e2-2435-4650-a336-1b4bee6bbc92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6728ab3b1c4f1baf6909e453b6e58c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:08.697359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1350/2720] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:08 2022] DELETE /identity/v3/users/2c6728ab3b1c4f1baf6909e453b6e58c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:09.146435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.147930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:09.161789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ab24-7ccc-43d6-ae06-610cb98832f3 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.219369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.220269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.220550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.220828 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] PATH_INFO: `/v3/projects/31c674ae326a499689fccfcb613fca46` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.222040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c674ae326a499689fccfcb613fca46)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.224041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.307329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:09.316535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c674ae326a499689fccfcb613fca46', 'request_id': 'req-2ee247e2-c471-411e-b738-eb376389b68c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:09.322379 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c674ae326a499689fccfcb613fca46', 'request_id': 'req-2ee247e2-c471-411e-b738-eb376389b68c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:09.425142 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c674ae326a499689fccfcb613fca46', 'request_id': 'req-2ee247e2-c471-411e-b738-eb376389b68c'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:09.436961 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] The token cache is being invalidate because project 31c674ae326a499689fccfcb613fca46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:09.437279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:09.438059 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-2ee247e2-c471-411e-b738-eb376389b68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:09.439860 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1371/2721] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:09 2022] DELETE /identity/v3/projects/31c674ae326a499689fccfcb613fca46 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:48:09.444904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.444904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.445410 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.464344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:09.465563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:09.524652 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1351/2722] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:09.532355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.532646 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.532924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.551231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:09.552408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:09.560725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1372/2723] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:09.564921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.575251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746b52-cc2f-4594-aebd-3aaf87be0c23 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.579989 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.580636 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.580920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.581226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.581949 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:48:09.582256 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.583884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.595894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1352/2724] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:48:09.599987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.609987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ad3d-d4a1-42e8-bf42-5681c6fafef6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.614798 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.615450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.615733 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.616010 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.616697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.618765 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.693182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1373/2725] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:48:09 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:09.702389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.707622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542863e-9949-4d89-a5b0-248444196fb9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.719389 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:09.763477 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.764146 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.764435 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.764716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.765433 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.767339 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.897416 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1353/2726] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:48:09 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:09.901908 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.912174 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b98683-0510-4276-b17b-53cb30e683c0 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.917081 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.917857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.918143 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.918420 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.919150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.920737 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.931866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1374/2727] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:09.935831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.946244 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ae82-2e5d-4962-ac61-7a3aa7109f2b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.951462 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.952120 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.952403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.952679 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:09.953434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:09.955015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7e304-ff70-4e97-8ad5-100be5619931 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:09.965709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1354/2728] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:09.969906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:09.980303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857998e5-1bae-4250-adf9-fb9851f240f1 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:09.985307 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:09.986201 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:09.986482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:09.986756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] PATH_INFO: `/v3/projects/a742732aebba4c65a279f453cbce5162/users/70c12c9e93814333890d01056637a0d2/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.001988 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a742732aebba4c65a279f453cbce5162, user_id=70c12c9e93814333890d01056637a0d2, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:10.004889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:10.103545 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1375/2729] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:09 2022] PUT /identity/v3/projects/a742732aebba4c65a279f453cbce5162/users/70c12c9e93814333890d01056637a0d2/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:10.115530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:10.115822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:10.116104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.143720 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] MFA Rules not processed for user `70c12c9e93814333890d01056637a0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:10.144927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:10.207591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1355/2730] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:10.219011 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:10.220716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:10.276321 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f64c6-6992-44bd-a1af-54c760a00d15 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:10.279968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:10.280599 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:10.280880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:10.281177 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.289367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:10.290782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a99d2-3391-4fa9-b329-3d67623aa7b5 service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:10.299542 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1376/2731] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:10.318565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:10.382661 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad738b-1271-4703-8d9e-5e778a557a3d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:10.387865 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:10.388729 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:10.389010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:10.389333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] PATH_INFO: `/v3/users/70c12c9e93814333890d01056637a0d2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.390551 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c12c9e93814333890d01056637a0d2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:10.392468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:10.492427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:10.586437 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c12c9e93814333890d01056637a0d2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:10.587129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c12c9e93814333890d01056637a0d2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:10.587864 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c12c9e93814333890d01056637a0d2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:10.595207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c12c9e93814333890d01056637a0d2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:10.607238 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c12c9e93814333890d01056637a0d2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:10.696988 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1356/2732] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:10 2022] DELETE /identity/v3/users/70c12c9e93814333890d01056637a0d2 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:10.701482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:10.701774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:10.702053 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.718771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:10.720056 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:10.776764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1377/2733] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:10.786174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:10.787567 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:10.844619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2a13-af9d-46ca-9713-9ad73921146e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:10.848161 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:10.848826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:10.849110 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:10.849419 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:10.858780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:10.860218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c329c5-d60b-473c-b0d0-4adc779d8f28 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:10.870142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1357/2734] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:48:11.293276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:11.294779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:11.302285 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:11.307883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da65dc-ff6a-4c42-ace8-2671d1c86fbb None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:11.312808 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:11.313716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.314006 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.314318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] PATH_INFO: `/v3/projects/a742732aebba4c65a279f453cbce5162` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.315536 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a742732aebba4c65a279f453cbce5162)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:11.317653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:11.428702 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a742732aebba4c65a279f453cbce5162', 'request_id': 'req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:11.434222 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a742732aebba4c65a279f453cbce5162', 'request_id': 'req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:11.508481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a742732aebba4c65a279f453cbce5162', 'request_id': 'req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:11.522445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] The token cache is being invalidate because project a742732aebba4c65a279f453cbce5162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:11.522736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:11.523371 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e68c645f-1e38-4c67-9d28-f7ea9863b0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:11.525541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1378/2735] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:11 2022] DELETE /identity/v3/projects/a742732aebba4c65a279f453cbce5162 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:11.530445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.530921 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.531021 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.548488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:11.552863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:11.608396 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1358/2736] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:11.670577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.670864 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.671139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.689411 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:11.690651 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:11.699257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1379/2737] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:11.703507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:11.714267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e559f-3d68-4e05-863a-2ec01d678662 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:11.718881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:11.719534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.719821 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.720098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.720822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:48:11.721131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:11.722780 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:11.734217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1359/2738] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:48:11.738443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:11.748242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0433c1-9fb7-4923-a438-16f23637afcd None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:11.752850 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:11.753781 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.754067 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.754343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.755077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:11.756972 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:11.849998 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1380/2739] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:48:11 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:11.854123 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:11.864304 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f62-1eb6-4cf2-80f1-2bfee50006f2 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:11.894778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:11.919811 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:11.920542 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:11.920828 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:11.921131 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:11.921839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:11.923732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:12.048053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1360/2740] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:48:11 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:48:12.052181 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:12.062880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7787-402b-4a24-bae4-b18c85ea6496 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:12.067517 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:12.068199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:12.068483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:12.068756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:12.069515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:12.071090 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:12.081614 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1381/2741] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:48:12.085466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:12.095450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d5bd5-f04e-45a8-ac4e-7d5835d0d412 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:12.100162 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:12.100816 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:12.101210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:12.101493 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:12.102210 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:12.104292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb5da8-058c-45fa-81b4-48ca37cc86a1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:12.114789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1361/2742] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:48:12.119569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:12.130039 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fb83-c327-40fe-abc7-ee65b61ed221 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:12.134962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:12.135831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:12.136112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:12.136385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] PATH_INFO: `/v3/projects/916170d2d41e43e69d105623dcc0193e/users/15de421106e842da8721a502a2076ad2/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:12.152639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=916170d2d41e43e69d105623dcc0193e, user_id=15de421106e842da8721a502a2076ad2, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:12.155319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:12.248866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1382/2743] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:12 2022] PUT /identity/v3/projects/916170d2d41e43e69d105623dcc0193e/users/15de421106e842da8721a502a2076ad2/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:12.260119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:12.260414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:12.260698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:12.288699 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] MFA Rules not processed for user `15de421106e842da8721a502a2076ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:12.289967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:12.358034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1362/2744] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:12.368772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:12.370524 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:12.426694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3838b-4374-4ed4-ab3d-331ca48729f9 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:12.430466 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:12.431180 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:12.431467 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:12.431745 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:12.440585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:12.442042 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8ad71-d0d0-4fec-8af3-1432565e5eee service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:12.451664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1383/2745] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:22.354952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:22.447442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:24.528181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:24.589004 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99c0a5-026b-4f75-85c9-a878c8ebec58 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:24.593572 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:24.594434 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:24.594717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:24.594993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] PATH_INFO: `/v3/users/15de421106e842da8721a502a2076ad2` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:24.596167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15de421106e842da8721a502a2076ad2)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:24.598129 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:24.815656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15de421106e842da8721a502a2076ad2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:24.816224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15de421106e842da8721a502a2076ad2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:24.816829 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15de421106e842da8721a502a2076ad2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:24.824115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15de421106e842da8721a502a2076ad2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:24.835792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15de421106e842da8721a502a2076ad2'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:24.899042 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1363/2746] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:48:24 2022] DELETE /identity/v3/users/15de421106e842da8721a502a2076ad2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:24.903795 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:24.904296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:24.904402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:24.921801 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:24.923002 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:24.984521 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1384/2747] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:24.994725 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:25.001104 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:25.059665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87e84-d6a7-4e32-bbb0-54cc25251f72 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:25.063365 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:25.064018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:25.064311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:25.064593 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:25.073996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:25.075366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816172c2-49b5-41a5-8767-f20b9d35e4c3 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:25.085466 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1364/2748] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:25.523099 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:25.526566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:25.554367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382f07-7894-44f7-900f-53f411f4a06e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:25.554367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:25.554367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:25.554367 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:25.554804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] PATH_INFO: `/v3/projects/916170d2d41e43e69d105623dcc0193e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:25.556269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916170d2d41e43e69d105623dcc0193e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:25.558486 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:25.735241 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916170d2d41e43e69d105623dcc0193e', 'request_id': 'req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:25.740657 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916170d2d41e43e69d105623dcc0193e', 'request_id': 'req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:25.806443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916170d2d41e43e69d105623dcc0193e', 'request_id': 'req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:25.818279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] The token cache is being invalidate because project 916170d2d41e43e69d105623dcc0193e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:48:25.818723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:25.819501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1a401a6c-d1fc-4b4e-88cd-b319b7ddebb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:48:25.821775 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1385/2749] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:48:25 2022] DELETE /identity/v3/projects/916170d2d41e43e69d105623dcc0193e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:48:25.826845 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:25.827411 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:25.827513 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:25.844558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:25.845784 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:25.862649 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:25.903905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1365/2750] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:48:25.913037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:25.913541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:25.913664 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:25.931774 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:25.932979 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:25.940957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1386/2751] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:25.945141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:25.956789 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ba3d-9e14-47c8-a384-3ddc4313246a None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:25.961440 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:25.962734 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:25.963014 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:25.963288 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:25.964010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:48:25.964320 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:25.966018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:25.977812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1366/2752] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:48:25.985540 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:25.995679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f01d-e255-46f2-81a8-bca6e74b711c None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.000530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.001238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.001531 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.001813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.002496 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.004663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.139590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1387/2753] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:48:25 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:48:26.143860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.154051 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc8c6-fe47-47c3-b618-2ebfad57ff3f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.211591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.212059 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.212345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.212620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.213354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.216534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.401342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1367/2754] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:48:26 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:48:26.405324 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.415402 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.415956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c48d-e198-4b3e-ac4b-8fe03fc49e50 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.418147 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:26.420753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.421444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.421726 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.422005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.422734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.424404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.435756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1388/2755] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:48:26.442197 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.453928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdae06f-777f-48c3-8939-958a7b85a319 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.459294 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.459987 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.460276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.460556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.461319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.463530 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.474515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1389/2756] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:48:26.478518 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.488043 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386a2e7-9f86-43af-ae04-aa95a28d2480 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.492668 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc85caf-d6a4-47e9-8d2e-43ed8f30629f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.492938 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.493605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.493891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.494168 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] PATH_INFO: `/v3/projects/8d364ea18c984a7888b0109368e56068/users/46c34777a21e4224afad443847d53fd7/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.498203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.498847 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.499130 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.499408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.501408 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:26.510844 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d364ea18c984a7888b0109368e56068, user_id=46c34777a21e4224afad443847d53fd7, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.513575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.514285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.515844 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40722d20-ee47-4be5-83a3-4bf934fc7dcd service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.526302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1368/2757] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:48:26.641497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1390/2758] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:48:26 2022] PUT /identity/v3/projects/8d364ea18c984a7888b0109368e56068/users/46c34777a21e4224afad443847d53fd7/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:48:26.653239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.653558 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.653932 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.681860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] MFA Rules not processed for user `46c34777a21e4224afad443847d53fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:26.683090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:26.701656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:26.749241 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1369/2759] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:26.761172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:26.762894 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:26.774242 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:26.824859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753fa78-bd63-474a-9b4b-e1e155873ec3 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:26.828556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '401f40d749db4643824d38db0cf533cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:26.829231 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:26.829520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:26.830485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:26.838585 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:26.840016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:26.849044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1391/2760] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:27.926095 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:27.927897 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:27.994625 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c69825-f6c5-4ad8-afc4-cdc94de6c242 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:27.998292 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:27.998938 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:27.999225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:27.999501 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:28.001343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:28.011885 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:28.013359 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd5f25-fafc-4c06-86b4-84655b413c73 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:28.021266 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1370/2761] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:28.095750 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:48:28.104866 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1612-c72a-419a-b89b-fb5f74b15a69 service glance] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:48:28.108912 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:48:28.109593 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:28.109896 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:28.110175 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:28.112041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:28.191736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:48:28.193268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb32a8c-f06a-485d-a1a5-914422da99ad service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:48:28.217234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1392/2762] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:48:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:48:38.017731 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=45 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:38.203049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=56 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:48:43.238491 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:43.238953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:43.239074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:43.257220 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] MFA Rules not processed for user `2b123a78349c4daba0502a4e9e98245f`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:43.258421 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:43.318863 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1371/2763] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:52.686194 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059a5b9-747d-4cbb-8932-36c13a15ba6f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:48:52.686735 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059a5b9-747d-4cbb-8932-36c13a15ba6f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:48:52.686855 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059a5b9-747d-4cbb-8932-36c13a15ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:48:52.704443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-9059a5b9-747d-4cbb-8932-36c13a15ba6f None None] MFA Rules not processed for user `f926d674e2dc4eb58d7fe6379b79310b`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:48:52.705734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-9059a5b9-747d-4cbb-8932-36c13a15ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:48:52.760687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1393/2764] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:48:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:48:53.307762 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=25 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:01.507399 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:01.571472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215843d9-b21d-4444-8e85-477d99f667d1 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:01.576442 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:01.577345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:01.577630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:01.577907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] PATH_INFO: `/v3/users/46c34777a21e4224afad443847d53fd7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:01.581286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c34777a21e4224afad443847d53fd7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:01.581286 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:01.804524 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c34777a21e4224afad443847d53fd7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:01.805243 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c34777a21e4224afad443847d53fd7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:01.805857 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c34777a21e4224afad443847d53fd7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:01.813452 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c34777a21e4224afad443847d53fd7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:01.827128 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c34777a21e4224afad443847d53fd7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:01.919635 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1372/2765] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:49:01 2022] DELETE /identity/v3/users/46c34777a21e4224afad443847d53fd7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:49:01.924184 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:01.924676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:01.924786 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:01.944443 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:49:01.945728 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:02.016504 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1394/2766] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:49:02.027246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:02.029688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:02.101587 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c84ae-503f-4368-83e2-30e5be91ab22 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:02.107020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:02.107020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:02.107020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:02.107020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:02.116529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:02.117975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:02.128644 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1373/2767] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:49:02.595532 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:02.597103 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:02.609676 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecd311-1e6d-4f02-834c-ae3724c82b3b None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:02.614303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:02.615160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:02.615471 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:02.615746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] PATH_INFO: `/v3/projects/8d364ea18c984a7888b0109368e56068` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:02.616920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d364ea18c984a7888b0109368e56068)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:02.618872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:02.735057 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d364ea18c984a7888b0109368e56068', 'request_id': 'req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:02.740782 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d364ea18c984a7888b0109368e56068', 'request_id': 'req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:02.801840 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d364ea18c984a7888b0109368e56068', 'request_id': 'req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:02.813758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] The token cache is being invalidate because project 8d364ea18c984a7888b0109368e56068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:49:02.814045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:02.814671 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1607c2b6-9084-470a-9a6a-45e73e1bb2b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:49:02.816237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1395/2768] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:49:02 2022] DELETE /identity/v3/projects/8d364ea18c984a7888b0109368e56068 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:49:04.265119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:04.268023 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:04.297936 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:04.336390 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265860a-9db5-49e9-8430-dbfc3f196046 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:04.399850 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:04.400739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:04.401293 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:04.401745 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:04.403884 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:04.466507 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:04.468066 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:04.477409 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1374/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:49:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:49:12.810588 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:12.952507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:12.953009 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:12.953485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:12.971434 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] MFA Rules not processed for user `917bfc7c68384536a6945af83308f789`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:49:12.972618 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:12.979982 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1396/2770] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:49:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:49:15.247815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=58 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:17.441851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:17.443283 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:17.621781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d380-2509-40fa-8adf-689588ccfcf8 service cinder] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:17.625347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:17.625992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:17.626278 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:17.626555 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:17.628329 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:17.696186 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:17.697645 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:17.709289 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1375/2771] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:49:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:49:18.816309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:18.828523 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5258c-6907-4dd1-aade-089dd5325d73 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:18.832154 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:18.832763 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:18.833045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:18.833348 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:18.835123 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:18.849260 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:18.850675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:18.861744 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1397/2772] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:49:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:49:27.704198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=54 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:28.856881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=9 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:49:39.030760 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:49:39.039633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd233d-603f-49ce-a3c2-1cba6d6f5fca service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:49:39.043398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:49:39.044055 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:49:39.044343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:49:39.044614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:49:39.046475 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:49:39.108008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:49:39.109451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:49:39.118915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1376/2773] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:49:49.114690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=30 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:50:03.780570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:03.782096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:03.854534 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725550-aaee-4b09-9fbb-1cd92553f10c service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:03.859076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:03.859955 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:03.860255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:03.860515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] PATH_INFO: `/v3/users/2b123a78349c4daba0502a4e9e98245f` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:03.861777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b123a78349c4daba0502a4e9e98245f)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:03.863784 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:04.060672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b123a78349c4daba0502a4e9e98245f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.061255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b123a78349c4daba0502a4e9e98245f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.061872 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b123a78349c4daba0502a4e9e98245f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.071309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b123a78349c4daba0502a4e9e98245f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.086359 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b123a78349c4daba0502a4e9e98245f'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.156452 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1398/2774] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:50:03 2022] DELETE /identity/v3/users/2b123a78349c4daba0502a4e9e98245f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:50:04.414486 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:04.427045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de616-2f86-49f9-9f7b-4afa283968c4 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:04.494413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:04.495282 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:04.495564 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:04.495839 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] PATH_INFO: `/v3/projects/e89c4c765f5243eaa314231429d20598` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:04.497201 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89c4c765f5243eaa314231429d20598)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:04.499115 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:04.641432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89c4c765f5243eaa314231429d20598', 'request_id': 'req-dbc1ba07-ac1b-48d2-a883-265e78b7d377'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.647488 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89c4c765f5243eaa314231429d20598', 'request_id': 'req-dbc1ba07-ac1b-48d2-a883-265e78b7d377'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.730398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89c4c765f5243eaa314231429d20598', 'request_id': 'req-dbc1ba07-ac1b-48d2-a883-265e78b7d377'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.746221 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] The token cache is being invalidate because project e89c4c765f5243eaa314231429d20598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:50:04.746518 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.747804 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-dbc1ba07-ac1b-48d2-a883-265e78b7d377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:50:04.749683 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1377/2775] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:50:04 2022] DELETE /identity/v3/projects/e89c4c765f5243eaa314231429d20598 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:50:11.889326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:11.890914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:11.980384 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32832198-5487-47ae-8ba7-15f9abfbe95b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:12.000442 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:50:12.059577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:12.059985 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:12.060268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:12.060598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:12.062468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:12.078756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:12.080490 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:12.093932 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1399/2776] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:50:14.739246 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:50:17.180523 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:17.181222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:17.181222 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:17.198325 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] MFA Rules not processed for user `f926d674e2dc4eb58d7fe6379b79310b`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:50:17.199568 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:17.253185 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1378/2777] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:50:18.214539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:18.216077 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:18.287997 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c7f79-8f40-467c-a994-eb917a42863a service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:18.293044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:18.293713 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:18.294063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:18.294375 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:18.296129 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:18.307602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:18.309343 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:18.319573 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1400/2778] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:50:21.402846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:21.404264 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:21.461482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b2cd7-98e6-428c-bbe5-12fdeced5f0c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:21.465036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:21.465718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:21.466000 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:21.466279 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:21.468078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:21.477910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:21.479311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:21.488016 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1379/2779] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:50:28.228901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:50:28.230435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:28.246491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9677e70-fecb-4952-a7b2-53dc54ef79f3 service placement] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:50:28.252082 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:50:28.252716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:50:28.253000 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:50:28.253312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:50:28.255831 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:50:28.267937 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:50:28.269520 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:50:28.281587 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1401/2780] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:50:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:50:31.484107 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=51 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:50:38.275688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=64 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:14.058165 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:14.059768 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:14.077191 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b4019-f0c0-4839-a2f1-3540b8ec27bd service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:14.083196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:14.084052 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:14.084194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:14.084395 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:14.086637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:14.100149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:14.101886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205ce1-34b3-4e77-9c0d-8dc75bc3a48d service nova] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:14.113267 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1380/2781] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:51:24.108998 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=6 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:29.470253 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:29.479830 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058f756-8471-44f0-b042-d7ddf53c6e19 service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:29.484026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:29.484878 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:29.485187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:29.485476 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:29.487264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:29.552389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:29.553869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:29.563623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1402/2782] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:51:31.031841 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:31.032761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:31.033048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:31.052803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] MFA Rules not processed for user `f926d674e2dc4eb58d7fe6379b79310b`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:51:31.054078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:31.065956 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1381/2783] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:51:31.570863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:31.580040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616831e5-7200-44fc-91c0-e29e0c7b36b0 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:31.583596 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:31.584239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:31.584519 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:31.584796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:31.586570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:31.596649 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:31.598111 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:31.606269 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1403/2784] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:51:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:51:34.158993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:34.160401 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:34.178069 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb918-495a-4375-b97c-d6fed643ca41 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:34.182977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:34.183622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:34.183904 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:34.184177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:34.186164 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:34.197959 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:34.199517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:34.210039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1382/2785] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:51:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:51:39.155117 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:39.156597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:39.171665 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2a3d0-89e7-4d8b-900a-b4c33488f466 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:39.176550 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:39.177502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:39.177771 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:39.178046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] PATH_INFO: `/v3/users/917bfc7c68384536a6945af83308f789` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:39.179258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] RBAC: Authorizing `identity:delete_user(user_id=917bfc7c68384536a6945af83308f789)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:39.181207 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:39.362193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917bfc7c68384536a6945af83308f789'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:39.362849 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917bfc7c68384536a6945af83308f789'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:39.363473 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917bfc7c68384536a6945af83308f789'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:39.370948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917bfc7c68384536a6945af83308f789'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:39.384748 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917bfc7c68384536a6945af83308f789'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:39.477069 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1404/2786] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:51:39 2022] DELETE /identity/v3/users/917bfc7c68384536a6945af83308f789 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:51:39.739141 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:39.752444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801762-41a2-49a8-bee1-c0b291fee92b service placement] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:39.813854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:39.814992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:39.815285 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:39.815570 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] PATH_INFO: `/v3/projects/c361a671e3fc43c8a536f6526dc530b1` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:39.816778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c361a671e3fc43c8a536f6526dc530b1)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:39.818716 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:39.996982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c361a671e3fc43c8a536f6526dc530b1', 'request_id': 'req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.002554 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c361a671e3fc43c8a536f6526dc530b1', 'request_id': 'req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.067997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c361a671e3fc43c8a536f6526dc530b1', 'request_id': 'req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.079851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] The token cache is being invalidate because project c361a671e3fc43c8a536f6526dc530b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:51:40.080138 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.080761 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.082314 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1383/2787] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:51:39 2022] DELETE /identity/v3/projects/c361a671e3fc43c8a536f6526dc530b1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:51:40.086444 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:40.087905 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:40.102435 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7211b-9f5f-4a87-b261-463575aa107c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:40.149393 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:40.158650 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:40.159512 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:40.159796 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:40.160071 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] PATH_INFO: `/v3/users/0a37cca180604534860037a2641b117e` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:40.161299 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a37cca180604534860037a2641b117e)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:40.163239 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:40.386584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a37cca180604534860037a2641b117e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.387149 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a37cca180604534860037a2641b117e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.390062 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a37cca180604534860037a2641b117e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.395416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a37cca180604534860037a2641b117e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.407104 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a37cca180604534860037a2641b117e'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:40.517237 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1405/2788] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:51:40 2022] DELETE /identity/v3/users/0a37cca180604534860037a2641b117e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:51:40.881272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:40.882697 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:40.898009 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4dd5-30ce-4299-9c78-5a1e1500fef1 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:40.954170 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:40.955022 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:40.955303 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:40.955579 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] PATH_INFO: `/v3/projects/1dfe50850f27465da257fe324ca5067e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:40.956984 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfe50850f27465da257fe324ca5067e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:40.958694 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:40.969854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:41.123815 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfe50850f27465da257fe324ca5067e', 'request_id': 'req-6dbe9f16-098c-4911-b61f-99a770b34d4c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:41.129321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfe50850f27465da257fe324ca5067e', 'request_id': 'req-6dbe9f16-098c-4911-b61f-99a770b34d4c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:41.236248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfe50850f27465da257fe324ca5067e', 'request_id': 'req-6dbe9f16-098c-4911-b61f-99a770b34d4c'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:41.247379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] The token cache is being invalidate because project 1dfe50850f27465da257fe324ca5067e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:51:41.253881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:41.253881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:51:41.253881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1384/2789] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:51:40 2022] DELETE /identity/v3/projects/1dfe50850f27465da257fe324ca5067e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:51:50.513514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:51.280252 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:15.646322 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:15.647760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:15.722605 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc8807-4ba5-4424-9fb5-33b51854868c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:15.785663 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:15.786309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:15.786590 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:15.786868 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:15.788668 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:15.845372 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:15.846959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:15.855891 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1406/2790] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:22.630696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:22.632940 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:22.689944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe9f16-098c-4911-b61f-99a770b34d4c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:22.693531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:22.694177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:22.694461 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:22.694738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:22.696533 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:22.758198 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:22.759596 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:22.769133 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1385/2791] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:23.945656 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:23.947216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:23.963597 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6884-2abd-4c9b-9e77-0458d59992f0 service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:23.979388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4ed458d984a51829114a73534fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:23.979388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:23.979388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:23.979388 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:23.980050 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:24.017408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=100 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:24.043814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:24.045624 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:24.055128 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1407/2792] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:30.418907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:30.420332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:30.433084 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76d519-58aa-4200-83eb-bdfec2846036 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:30.437746 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:30.438682 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:30.438977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:30.439251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] PATH_INFO: `/v3/users/df735fc7405643f285f529ddfa611064` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:30.440449 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df735fc7405643f285f529ddfa611064)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:30.442447 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:30.637125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df735fc7405643f285f529ddfa611064'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:30.637738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df735fc7405643f285f529ddfa611064'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:30.638345 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df735fc7405643f285f529ddfa611064'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:30.645217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df735fc7405643f285f529ddfa611064'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:30.656939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df735fc7405643f285f529ddfa611064'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:30.726967 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1386/2793] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:52:30 2022] DELETE /identity/v3/users/df735fc7405643f285f529ddfa611064 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:30.974743 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:30.985607 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca1813-81fa-4032-9fd4-a164f197a81d service nova] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:31.043080 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:31.044869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:31.044869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:31.044869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] PATH_INFO: `/v3/projects/8fa20247a7af46d0b9660d4cd026f552` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:31.045734 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa20247a7af46d0b9660d4cd026f552)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:31.047632 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:31.187811 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa20247a7af46d0b9660d4cd026f552', 'request_id': 'req-10475893-85b2-4c84-81dd-01c64e44ef9d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.193814 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa20247a7af46d0b9660d4cd026f552', 'request_id': 'req-10475893-85b2-4c84-81dd-01c64e44ef9d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.298178 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa20247a7af46d0b9660d4cd026f552', 'request_id': 'req-10475893-85b2-4c84-81dd-01c64e44ef9d'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.309738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] The token cache is being invalidate because project 8fa20247a7af46d0b9660d4cd026f552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:52:31.310026 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.310675 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.312232 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1408/2794] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:52:30 2022] DELETE /identity/v3/projects/8fa20247a7af46d0b9660d4cd026f552 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:31.316194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:31.318247 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:31.331272 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e21b-9a62-4fe5-8e1d-6abe2e388f03 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:31.350018 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:31.390001 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:31.391062 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:31.391436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:31.391823 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] PATH_INFO: `/v3/users/2da99a3d46c34811b3d895f24e69b3a8` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:31.393136 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da99a3d46c34811b3d895f24e69b3a8)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:31.395797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:31.623445 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da99a3d46c34811b3d895f24e69b3a8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.624112 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da99a3d46c34811b3d895f24e69b3a8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.624607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da99a3d46c34811b3d895f24e69b3a8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.632035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da99a3d46c34811b3d895f24e69b3a8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.643670 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da99a3d46c34811b3d895f24e69b3a8'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:31.732836 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1387/2795] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:52:31 2022] DELETE /identity/v3/users/2da99a3d46c34811b3d895f24e69b3a8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:32.025510 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:32.027141 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:32.042064 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10475893-85b2-4c84-81dd-01c64e44ef9d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:32.100225 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:32.101284 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:32.102317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:32.102317 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] PATH_INFO: `/v3/projects/f699e58f71d24bb7b4a700b3b3586180` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:32.102914 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f699e58f71d24bb7b4a700b3b3586180)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:32.104803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:32.111570 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:32.206366 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f699e58f71d24bb7b4a700b3b3586180', 'request_id': 'req-7e3942c3-6355-4353-b3e6-d46a16610bff'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:32.212298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f699e58f71d24bb7b4a700b3b3586180', 'request_id': 'req-7e3942c3-6355-4353-b3e6-d46a16610bff'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:32.287360 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f699e58f71d24bb7b4a700b3b3586180', 'request_id': 'req-7e3942c3-6355-4353-b3e6-d46a16610bff'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:32.298430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] The token cache is being invalidate because project f699e58f71d24bb7b4a700b3b3586180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:52:32.298719 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:32.299340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:32.301014 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1409/2796] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:52:32 2022] DELETE /identity/v3/projects/f699e58f71d24bb7b4a700b3b3586180 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:41.730472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:42.295551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:47.226049 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:47.227563 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:47.283798 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff47cd9-fe30-403a-b059-049e4d322a54 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:47.330334 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:47.330982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:47.331265 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:47.331543 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:47.333886 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:47.395797 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:47.397166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:47.406837 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1388/2797] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:51.112409 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:51.114670 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:51.127569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3942c3-6355-4353-b3e6-d46a16610bff admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:51.133021 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:51.133926 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:51.134213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:51.134491 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] PATH_INFO: `/v3/users/f926d674e2dc4eb58d7fe6379b79310b` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:51.135688 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f926d674e2dc4eb58d7fe6379b79310b)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:51.137751 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:51.372928 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f926d674e2dc4eb58d7fe6379b79310b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:51.376911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f926d674e2dc4eb58d7fe6379b79310b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:51.376911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f926d674e2dc4eb58d7fe6379b79310b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:51.383185 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f926d674e2dc4eb58d7fe6379b79310b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:51.395863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-e51c709f-7112-43a8-9fd3-c5485931d345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f926d674e2dc4eb58d7fe6379b79310b'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:51.556027 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1410/2798] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:52:51 2022] DELETE /identity/v3/users/f926d674e2dc4eb58d7fe6379b79310b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:51.859465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:51.867561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff60619-8504-4772-ada6-d68a8dd71e4f service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:51.923807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:51.924658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:51.924942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:51.925251 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] PATH_INFO: `/v3/projects/35438fec2e37438e99201870e0d650af` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:51.926951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35438fec2e37438e99201870e0d650af)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:51.928852 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:51.936608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=100 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:52.135794 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35438fec2e37438e99201870e0d650af', 'request_id': 'req-9eaf8570-5943-465c-88f5-9bcf94144cb4'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:52.141194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35438fec2e37438e99201870e0d650af', 'request_id': 'req-9eaf8570-5943-465c-88f5-9bcf94144cb4'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:52.246180 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35438fec2e37438e99201870e0d650af', 'request_id': 'req-9eaf8570-5943-465c-88f5-9bcf94144cb4'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:52.258860 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] The token cache is being invalidate because project 35438fec2e37438e99201870e0d650af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:52:52.259153 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:52.259820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-9eaf8570-5943-465c-88f5-9bcf94144cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:52.261431 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1389/2799] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:52:51 2022] DELETE /identity/v3/projects/35438fec2e37438e99201870e0d650af => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:52.266311 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.266823 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.266945 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.284711 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:52:52.285944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:52.343851 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1411/2800] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:52:52.351776 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.352067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.352374 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.371342 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:52:52.372403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:52.383090 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1390/2801] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:52:52.387515 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.397794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41505d1-8ccc-4f30-9b4d-159b75782971 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.402459 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.403114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.403400 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.403679 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.404412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:52:52.404721 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.406398 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:52.418645 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1412/2802] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:52:52.432606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.442236 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b4d9a-dc98-41dd-85c8-aa6278946a46 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.446951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.447608 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.447893 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.448172 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.448866 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.450800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:52.550171 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1391/2803] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:52:52 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:52:52.554261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.564255 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fb4bd-444f-4c4f-8b91-fca41804df78 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.620906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.621610 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.621934 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.622228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.622930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.624913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:52.631787 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:52.787951 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1413/2804] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:52:52 2022] POST /identity/v3/users => generated 303 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:52:52.792407 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.802637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd86a6-d78e-4990-8f57-154820fd64d4 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.807190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.807969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.808254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.808530 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.809284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.810958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:52.821274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1392/2805] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:52:52.825120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.835635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4facea-2cb2-4667-a600-1e7daee100c7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.840228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.840881 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.841188 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.841480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.842205 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.843789 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a272b-413d-4ed3-80de-19c6a559deb0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:52.856464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1414/2806] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:52:52.861929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:52.872877 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829c08-cb10-470e-8022-2df2be5f536f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:52.877764 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:52.878653 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:52.878944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:52.879223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] PATH_INFO: `/v3/projects/eab9ba8d0ca04180a416e3562ba28106/users/d0bd81daa42d4519acdfc25286d790da/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:52.894344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab9ba8d0ca04180a416e3562ba28106, user_id=d0bd81daa42d4519acdfc25286d790da, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:52.896997 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:53.018002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1393/2807] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:52:52 2022] PUT /identity/v3/projects/eab9ba8d0ca04180a416e3562ba28106/users/d0bd81daa42d4519acdfc25286d790da/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:53.023469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a2b0-a80d-4904-b1bf-46321755d7a5 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:53.023764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a2b0-a80d-4904-b1bf-46321755d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:53.024047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a2b0-a80d-4904-b1bf-46321755d7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:53.041685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-43f5a2b0-a80d-4904-b1bf-46321755d7a5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:52:53.042883 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-43f5a2b0-a80d-4904-b1bf-46321755d7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:53.100641 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1415/2808] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:52:53.110177 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:53.111958 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:53.168324 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67951a8-06d9-48fa-bdc0-0f21b74407eb admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:53.172149 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:53.172792 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:53.173077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:53.173392 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:53.182999 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:53.184414 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:53.193654 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1394/2809] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:54.970131 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:54.970658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:54.970790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:54.997755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] MFA Rules not processed for user `d0bd81daa42d4519acdfc25286d790da`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:52:54.998991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:55.067046 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1416/2810] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:52:55.072085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.089082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713c8a-4046-4622-9bdd-0ee967fd4ef1 service neutron] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.089082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.089082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.089082 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.089781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.089781 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.090907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.251077 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1395/2811] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:52:55 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:52:55.255480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.266533 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a5482-c63d-4bb9-ad7d-d30b259a421e None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.321813 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.322404 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.322687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.322968 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.323653 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.325635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.522096 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:55.528072 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1417/2812] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:52:55 2022] POST /identity/v3/users => generated 309 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:52:55.532088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.540995 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:55.543218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6b54c-add5-4adc-8379-5b96ff4a321d admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.547833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.548495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.548809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.549089 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.549851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.551441 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.561801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1396/2813] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:52:55.565785 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.575777 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f780e-3ee6-47e3-8e63-1cab3396a2b5 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.580382 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.581300 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.581617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.581857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] PATH_INFO: `/v3/projects/ab818aa07dce44838ddba066e5139ee7/users/03aa16fcc2d443d391ee8e11121b28b7/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.596709 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab818aa07dce44838ddba066e5139ee7, user_id=03aa16fcc2d443d391ee8e11121b28b7, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.599936 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.684290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1418/2814] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:52:55 2022] PUT /identity/v3/projects/ab818aa07dce44838ddba066e5139ee7/users/03aa16fcc2d443d391ee8e11121b28b7/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:55.688363 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.698534 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ff31f-7b4e-478d-9895-c80f565577aa admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.754005 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.754920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.755203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.755479 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.756207 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.757832 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f06e7-7780-4e2d-919d-6132e347a650 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.767413 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1397/2815] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:52:55.772086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:55.782977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332541-102e-41d7-8d8e-331b83253513 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:55.788193 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:55.789094 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:55.791598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:55.791598 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] PATH_INFO: `/v3/projects/ab818aa07dce44838ddba066e5139ee7/users/03aa16fcc2d443d391ee8e11121b28b7/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:55.791817 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab818aa07dce44838ddba066e5139ee7, user_id=03aa16fcc2d443d391ee8e11121b28b7, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:55.794385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:55.804481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1419/2816] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:52:55 2022] PUT /identity/v3/projects/ab818aa07dce44838ddba066e5139ee7/users/03aa16fcc2d443d391ee8e11121b28b7/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:52:57.465468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:57.466398 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:57.466687 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:57.494258 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] MFA Rules not processed for user `03aa16fcc2d443d391ee8e11121b28b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:52:57.495500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:57.565302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1398/2817] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:52:58.263325 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:58.265234 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:58.336508 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abaee9-1936-4f89-9498-c3a10fc3910e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:58.341544 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:58.342196 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:58.342485 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:58.342911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:58.344810 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:52:58.356468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:58.358187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:58.368703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1420/2818] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:52:59.198765 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:52:59.265620 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc2c6-fc60-4960-ae91-4b84c5d3ad95 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:52:59.270140 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:52:59.271015 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:52:59.271299 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:52:59.271571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] PATH_INFO: `/v3/users/d0bd81daa42d4519acdfc25286d790da` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:52:59.272766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0bd81daa42d4519acdfc25286d790da)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:52:59.274727 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:52:59.290902 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:52:59.588444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0bd81daa42d4519acdfc25286d790da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:59.589125 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0bd81daa42d4519acdfc25286d790da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:59.589809 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0bd81daa42d4519acdfc25286d790da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:59.596807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0bd81daa42d4519acdfc25286d790da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:59.609274 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0bd81daa42d4519acdfc25286d790da'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:52:59.706459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1399/2819] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:52:59 2022] DELETE /identity/v3/users/d0bd81daa42d4519acdfc25286d790da => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:00.030729 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:00.041463 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51e2ef-5a52-4d45-adef-a5b5d46c53ec service placement] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:00.099976 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:00.100839 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:00.101121 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:00.101428 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] PATH_INFO: `/v3/projects/eab9ba8d0ca04180a416e3562ba28106` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:00.102620 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab9ba8d0ca04180a416e3562ba28106)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:00.104563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:00.252084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab9ba8d0ca04180a416e3562ba28106', 'request_id': 'req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.254087 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:00.258308 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab9ba8d0ca04180a416e3562ba28106', 'request_id': 'req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.360447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab9ba8d0ca04180a416e3562ba28106', 'request_id': 'req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.373725 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] The token cache is being invalidate because project eab9ba8d0ca04180a416e3562ba28106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:00.374016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.374640 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.376213 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1421/2820] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:00 2022] DELETE /identity/v3/projects/eab9ba8d0ca04180a416e3562ba28106 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:00.380557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:00.382081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:00.394918 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede67b-754e-4b5c-bb00-eea2deb406ca admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:00.454922 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:00.456038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:00.456328 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:00.456606 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] PATH_INFO: `/v3/users/03aa16fcc2d443d391ee8e11121b28b7` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:00.458378 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03aa16fcc2d443d391ee8e11121b28b7)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:00.460315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:00.691101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03aa16fcc2d443d391ee8e11121b28b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.692332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03aa16fcc2d443d391ee8e11121b28b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.692939 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03aa16fcc2d443d391ee8e11121b28b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.699851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03aa16fcc2d443d391ee8e11121b28b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.711992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d616848d-bb15-4d1f-9f0d-5d37cf9d39fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03aa16fcc2d443d391ee8e11121b28b7'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:00.804010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1400/2821] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:00 2022] DELETE /identity/v3/users/03aa16fcc2d443d391ee8e11121b28b7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:01.105633 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.107086 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:01.120470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecbbe-a18e-48c4-bb5f-088b0786c0bb admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.178415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.179522 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.179803 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.180212 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] PATH_INFO: `/v3/projects/ab818aa07dce44838ddba066e5139ee7` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.181440 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab818aa07dce44838ddba066e5139ee7)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.183412 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:01.295832 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab818aa07dce44838ddba066e5139ee7', 'request_id': 'req-eb5784a7-5037-4131-9a7e-4565e0b17717'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:01.301358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab818aa07dce44838ddba066e5139ee7', 'request_id': 'req-eb5784a7-5037-4131-9a7e-4565e0b17717'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:01.406157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab818aa07dce44838ddba066e5139ee7', 'request_id': 'req-eb5784a7-5037-4131-9a7e-4565e0b17717'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:01.417861 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] The token cache is being invalidate because project ab818aa07dce44838ddba066e5139ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80347) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:01.418148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:01.418767 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-eb5784a7-5037-4131-9a7e-4565e0b17717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:01.420297 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1422/2822] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:01 2022] DELETE /identity/v3/projects/ab818aa07dce44838ddba066e5139ee7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:01.425436 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.425965 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.426581 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.444718 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:01.446048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:01.454669 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:01.505028 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1401/2823] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:01.512478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.512764 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.513041 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.531268 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:01.543312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:01.543312 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1423/2824] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:01.546103 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.556332 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc942f-4bd2-485e-b65d-a86152e3cd42 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.560976 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.561665 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.561951 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.562228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] PATH_INFO: `/v3/domains` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.562946 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80346) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:53:01.563255 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.564910 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:01.576882 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1402/2825] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:53:01.581037 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.594716 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af620088-d540-467d-9de8-2d94d46e5fb7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.595018 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.595661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.595948 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.596228 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.596933 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.599483 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:01.721790 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1424/2826] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:01 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:01.726341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.736347 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9dbc-8362-47ba-a4a8-694a4caee892 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.791531 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.792192 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.792478 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.792755 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.793482 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.795380 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:01.935228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1403/2827] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:01 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:01.939279 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.949592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95c9f6-a789-451b-90f5-03865ed222aa admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.954582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.955243 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.955553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.955833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.956564 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.958703 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:01.970133 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1425/2828] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:01.974561 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:01.984499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680eeed-619c-4ba3-9ff7-219d8faab67c admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:01.989235 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:01.989891 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:01.990181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:01.990465 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:01.991193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:01.992800 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e39a6b-aca4-4ec2-bdca-ae82f270aee4 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.002701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1404/2829] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:02.006904 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.017049 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ea73b-7ddb-4d6a-9aa2-3d904c1782a7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.022150 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.023015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.023295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.023566 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] PATH_INFO: `/v3/projects/2bacb17403914b589e6fe736a6f5461f/users/798af61b90ca486ba783e21ddf4950fb/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.038430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bacb17403914b589e6fe736a6f5461f, user_id=798af61b90ca486ba783e21ddf4950fb, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.041044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.169107 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1426/2830] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:02 2022] PUT /identity/v3/projects/2bacb17403914b589e6fe736a6f5461f/users/798af61b90ca486ba783e21ddf4950fb/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:53:02.179341 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.179630 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.179909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.205619 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] MFA Rules not processed for user `798af61b90ca486ba783e21ddf4950fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:02.206806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:02.268386 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1405/2831] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:02.273369 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.282226 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:02.334924 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564be261-211c-4b10-85f8-f3f630a737c6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.339551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.340199 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.340480 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.340755 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.341472 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.343358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.490647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1427/2832] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:02 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:02.494747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.506017 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958402f-c5c9-423d-86db-48073c3e9054 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.510975 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:02.560208 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.560806 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.561097 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.561403 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.562142 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.564053 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.721565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1406/2833] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:02 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:02.725717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.736152 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba968ff1-7026-4616-bdc0-3d893c5ab864 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.741415 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.742074 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.742361 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.742638 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.743397 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.745092 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.756413 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1428/2834] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:02.760854 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.770726 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb6b94-f051-464d-b83c-9ed80a4af5a7 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.775343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.776215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.776498 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.776775 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] PATH_INFO: `/v3/projects/f95743cc129a48269369209489ddc2b5/users/110d55472eb945ebb21abf1e608495a5/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.791375 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95743cc129a48269369209489ddc2b5, user_id=110d55472eb945ebb21abf1e608495a5, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.796034 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:02.918217 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1407/2835] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:02 2022] PUT /identity/v3/projects/f95743cc129a48269369209489ddc2b5/users/110d55472eb945ebb21abf1e608495a5/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:53:02.922303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:02.933047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c7f01-7159-4124-aff1-f743494a32ba admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:02.988258 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:02.989469 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:02.989758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:02.990034 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:02.990858 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:02.992461 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf34f-e39b-4a7f-af66-38833e7f44a0 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:03.002135 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1429/2836] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:03.006952 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:03.017108 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7767-ff2a-4b8c-9517-8b2d04afebb9 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:03.021785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:03.022658 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:03.022944 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:03.023223 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] PATH_INFO: `/v3/projects/f95743cc129a48269369209489ddc2b5/users/110d55472eb945ebb21abf1e608495a5/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:03.025239 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95743cc129a48269369209489ddc2b5, user_id=110d55472eb945ebb21abf1e608495a5, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:03.027849 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:03.038041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1408/2837] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:03 2022] PUT /identity/v3/projects/f95743cc129a48269369209489ddc2b5/users/110d55472eb945ebb21abf1e608495a5/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:03.048999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:03.049326 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:03.049615 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:03.076956 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] MFA Rules not processed for user `110d55472eb945ebb21abf1e608495a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:03.078215 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:03.146535 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1430/2838] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:03.157851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:03.159515 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:03.216263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f2ba-8055-490e-bdcf-a91d47aa73d8 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:03.219909 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:03.220546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:03.220825 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:03.221099 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:03.230565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:03.231977 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7902-5a6c-4629-9f0e-91ba814dc2c9 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:03.241831 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1409/2839] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:04.297899 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:04.302044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:04.358482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cff87-571c-448d-9099-ed3c56be8ee6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:04.363044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:04.363909 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:04.364191 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:04.364468 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] PATH_INFO: `/v3/users/798af61b90ca486ba783e21ddf4950fb` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:04.365669 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=798af61b90ca486ba783e21ddf4950fb)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:04.367562 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:04.602063 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798af61b90ca486ba783e21ddf4950fb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:04.602779 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798af61b90ca486ba783e21ddf4950fb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:04.603406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798af61b90ca486ba783e21ddf4950fb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:04.610589 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798af61b90ca486ba783e21ddf4950fb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:04.622217 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798af61b90ca486ba783e21ddf4950fb'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:04.749580 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1431/2840] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:04 2022] DELETE /identity/v3/users/798af61b90ca486ba783e21ddf4950fb => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:04.754813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:04.755276 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:04.755381 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:04.772469 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:04.773698 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:04.832833 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1410/2841] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:04.842565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:04.843977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:04.902482 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1829-4914-4370-98cf-b32e6606f0dc admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:04.906738 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:04.907507 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:04.907929 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:04.908216 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:04.917833 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:04.919256 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:04.928565 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1432/2842] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:05.394340 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:05.395803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:05.409373 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:05.410035 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f465828-6e95-4e71-97cc-6af16eadc2b3 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:05.419047 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:05.419881 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:05.420225 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:05.420599 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] PATH_INFO: `/v3/projects/2bacb17403914b589e6fe736a6f5461f` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:05.422723 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bacb17403914b589e6fe736a6f5461f)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:05.426923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:05.577008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bacb17403914b589e6fe736a6f5461f', 'request_id': 'req-2afcc09e-55bb-47df-9d56-432e192f868a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:05.582671 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bacb17403914b589e6fe736a6f5461f', 'request_id': 'req-2afcc09e-55bb-47df-9d56-432e192f868a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:05.735923 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bacb17403914b589e6fe736a6f5461f', 'request_id': 'req-2afcc09e-55bb-47df-9d56-432e192f868a'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:05.749586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] The token cache is being invalidate because project 2bacb17403914b589e6fe736a6f5461f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:05.749875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:05.750586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:05.752187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1411/2843] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:05 2022] DELETE /identity/v3/projects/2bacb17403914b589e6fe736a6f5461f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:05.756718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:05.767093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5675-0485-451d-bd72-585f15a54349 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:05.793084 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:05.833975 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:05.834852 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:05.835164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:05.835416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] PATH_INFO: `/v3/users/110d55472eb945ebb21abf1e608495a5` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:05.836622 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110d55472eb945ebb21abf1e608495a5)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:05.838672 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:06.171447 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110d55472eb945ebb21abf1e608495a5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.172055 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110d55472eb945ebb21abf1e608495a5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.172658 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110d55472eb945ebb21abf1e608495a5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.179582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110d55472eb945ebb21abf1e608495a5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.191093 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-774d31a9-3daf-411f-8db6-11e8eb957166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110d55472eb945ebb21abf1e608495a5'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.302825 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1433/2844] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:05 2022] DELETE /identity/v3/users/110d55472eb945ebb21abf1e608495a5 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:06.756609 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:06.758098 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:06.771929 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc09e-55bb-47df-9d56-432e192f868a admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:06.829343 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:06.830205 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:06.830622 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:06.830907 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] PATH_INFO: `/v3/projects/f95743cc129a48269369209489ddc2b5` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:06.832119 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95743cc129a48269369209489ddc2b5)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:06.834600 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:06.991366 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95743cc129a48269369209489ddc2b5', 'request_id': 'req-a7b868d8-f8e9-40a9-b80b-decbef843c40'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:06.996770 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95743cc129a48269369209489ddc2b5', 'request_id': 'req-a7b868d8-f8e9-40a9-b80b-decbef843c40'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:07.100803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95743cc129a48269369209489ddc2b5', 'request_id': 'req-a7b868d8-f8e9-40a9-b80b-decbef843c40'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:07.112517 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] The token cache is being invalidate because project f95743cc129a48269369209489ddc2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:07.112802 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:07.113505 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-a7b868d8-f8e9-40a9-b80b-decbef843c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:07.115203 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1412/2845] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:06 2022] DELETE /identity/v3/projects/f95743cc129a48269369209489ddc2b5 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:07.122145 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.122652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.122772 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.146020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:07.147408 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:07.209378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1434/2846] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:07.217371 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.217662 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.217941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.236025 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:07.237254 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:07.245576 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1413/2847] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:07.249727 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.260134 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9090bb-d451-4074-b7e6-3e6fbe7f2da7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.264661 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.265340 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.265623 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.265897 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.266617 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:53:07.266923 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.268502 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.279834 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1435/2848] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:53:07.284080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.293284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7aaa1a-1186-47be-b7da-559051aa5f9c None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.297945 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.298589 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.298873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.299152 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.299834 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.302257 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.310427 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:07.390370 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1414/2849] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:07.394514 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.404705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561a505-d78e-45ac-b478-73c840dbb6b6 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.433892 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:07.461493 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.462211 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.462498 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.462778 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.463478 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.465635 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.598848 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1436/2850] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:07.602927 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.613031 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b02595-0e63-4314-8eca-9f2705b22b8e admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.618080 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.618738 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.619020 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.619297 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.620067 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.621735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.632002 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1415/2851] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:07.635747 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.645991 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff68b-594f-4ae0-9c12-4c81dc8fdd7d admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.650647 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.651298 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.651582 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.651862 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.652586 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.654209 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32911-5393-4676-aef2-eb136aedda12 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.665354 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1437/2852] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:07.669336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.679733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b177901-5a9d-4b90-9df5-3fccc22bcf15 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.695159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.695159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.695159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.695159 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] PATH_INFO: `/v3/projects/5d88aeb6cc94469bad9dabc45b6f6eac/users/1d1693f479a84353a61408c88cfb2ba0/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.701444 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d88aeb6cc94469bad9dabc45b6f6eac, user_id=1d1693f479a84353a61408c88cfb2ba0, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.704194 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:07.777614 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1416/2853] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:07 2022] PUT /identity/v3/projects/5d88aeb6cc94469bad9dabc45b6f6eac/users/1d1693f479a84353a61408c88cfb2ba0/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:07.788556 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.788838 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.789120 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.816378 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] MFA Rules not processed for user `1d1693f479a84353a61408c88cfb2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:07.817736 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:07.883962 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1438/2854] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:07.888591 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:07.950167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36965b5-51c1-4744-97e9-bb4c02cae13f admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:07.955228 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:07.955928 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:07.956215 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:07.956495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:07.957730 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:07.959624 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.047474 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1417/2855] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:07 2022] POST /identity/v3/projects => generated 401 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:08.051959 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.062898 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d205b-6707-4956-9161-be3a762d6f6f None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.119352 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.120019 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.120303 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.120723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.121462 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.123387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.232606 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1439/2856] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:08 2022] POST /identity/v3/users => generated 307 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:08.236571 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.247801 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc19127-fa0c-4090-89d7-542f618b68fb admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.252388 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.253044 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.253352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.253637 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.255190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.256996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.267577 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1418/2857] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:08.271572 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.281718 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4854d06-5852-4821-8997-79c2bd39f86e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.286424 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.287373 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.287659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.287942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] PATH_INFO: `/v3/projects/ad198466643d436cab7134782f64069e/users/20f1c72a6a5a45eb97a208b425089700/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.303179 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad198466643d436cab7134782f64069e, user_id=20f1c72a6a5a45eb97a208b425089700, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.305889 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.386338 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1440/2858] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:08 2022] PUT /identity/v3/projects/ad198466643d436cab7134782f64069e/users/20f1c72a6a5a45eb97a208b425089700/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:08.390167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.400154 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472201-79a4-4d25-ba14-46b8765b2822 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.456273 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.456942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.457263 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.457546 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.458277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.459875 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77cf8-552a-478b-b6be-712862065a19 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.469574 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1419/2859] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:08.474124 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.480880 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:08.484639 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7746c-f2a2-4822-804f-856b608bcb8e admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.489992 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.490876 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.491160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.491436 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] PATH_INFO: `/v3/projects/ad198466643d436cab7134782f64069e/users/20f1c72a6a5a45eb97a208b425089700/roles/e92eaebf304e4ec6afef99f44d30b06d` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.493470 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad198466643d436cab7134782f64069e, user_id=20f1c72a6a5a45eb97a208b425089700, role_id=e92eaebf304e4ec6afef99f44d30b06d)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.496156 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.506584 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1441/2860] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:08 2022] PUT /identity/v3/projects/ad198466643d436cab7134782f64069e/users/20f1c72a6a5a45eb97a208b425089700/roles/e92eaebf304e4ec6afef99f44d30b06d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:53:08.567651 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.567941 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.568226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.594539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] MFA Rules not processed for user `20f1c72a6a5a45eb97a208b425089700`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:08.595751 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:08.607705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:08.662733 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1420/2861] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:08.673445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.675151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:08.732551 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7359b2-ce92-4ca0-a600-1f50a1feb4d7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.736301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.736869 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.737172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.737465 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:08.746953 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:08.750020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:08.758990 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1442/2862] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:08.970926 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:08.980623 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5b0a6-d3f4-4bc8-b833-8be1e8d524c4 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:08.984930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:08.984930 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:08.985190 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:08.985311 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:09.041102 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:09.042485 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab121c9-1f16-4c79-b0bd-6a6fa409abc4 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:09.051807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1421/2863] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:14.540870 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:14.602893 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ccc2a-11a0-458f-9e69-93d16630f7de service cinder] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:14.607410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:14.608290 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:14.608577 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:14.608854 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] PATH_INFO: `/v3/users/1d1693f479a84353a61408c88cfb2ba0` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:14.610076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1693f479a84353a61408c88cfb2ba0)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:14.614148 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:14.895804 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1693f479a84353a61408c88cfb2ba0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:14.896387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1693f479a84353a61408c88cfb2ba0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:14.896993 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1693f479a84353a61408c88cfb2ba0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:14.904114 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1693f479a84353a61408c88cfb2ba0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:14.917261 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1693f479a84353a61408c88cfb2ba0'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.006047 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1443/2864] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:14 2022] DELETE /identity/v3/users/1d1693f479a84353a61408c88cfb2ba0 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:15.010500 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:15.010972 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:15.011078 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:15.028758 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:15.029992 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:15.087565 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1422/2865] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:53:15.097172 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:15.099102 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:15.113971 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:15.159697 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abfd6a-bba9-4e14-b476-e2a70a3f8f88 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:15.162342 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:15.163015 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:15.163295 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:15.163569 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:15.172760 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:15.174265 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:15.184901 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1444/2866] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:15.608605 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:15.610969 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:15.630874 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba4ec1-7a52-4b91-a903-92404015ebba None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:15.636224 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:15.636990 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:15.637352 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:15.637709 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] PATH_INFO: `/v3/projects/5d88aeb6cc94469bad9dabc45b6f6eac` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:15.638822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d88aeb6cc94469bad9dabc45b6f6eac)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:15.641348 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:15.769747 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:15.773346 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d88aeb6cc94469bad9dabc45b6f6eac', 'request_id': 'req-8b4babc9-e9cd-459c-92de-096505ee7b06'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.778778 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d88aeb6cc94469bad9dabc45b6f6eac', 'request_id': 'req-8b4babc9-e9cd-459c-92de-096505ee7b06'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.919719 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d88aeb6cc94469bad9dabc45b6f6eac', 'request_id': 'req-8b4babc9-e9cd-459c-92de-096505ee7b06'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.931041 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] The token cache is being invalidate because project 5d88aeb6cc94469bad9dabc45b6f6eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:15.931330 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.931996 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:15.933557 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1423/2867] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:15 2022] DELETE /identity/v3/projects/5d88aeb6cc94469bad9dabc45b6f6eac => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:15.937705 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:15.947942 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09db00-df77-4ad2-9303-1e692276ec98 service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:16.004717 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:16.005609 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:16.005890 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:16.006164 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] PATH_INFO: `/v3/users/20f1c72a6a5a45eb97a208b425089700` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:16.007339 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f1c72a6a5a45eb97a208b425089700)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:16.009270 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:16.315655 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f1c72a6a5a45eb97a208b425089700'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:16.316319 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f1c72a6a5a45eb97a208b425089700'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:16.316920 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f1c72a6a5a45eb97a208b425089700'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:16.323944 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f1c72a6a5a45eb97a208b425089700'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:16.336276 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-1f75317e-e387-4f08-84d5-8ee3ca3970f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f1c72a6a5a45eb97a208b425089700'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:16.426266 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1445/2868] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:15 2022] DELETE /identity/v3/users/20f1c72a6a5a45eb97a208b425089700 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:16.875785 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:16.877539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:16.891166 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4babc9-e9cd-459c-92de-096505ee7b06 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:16.949291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:16.950174 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:16.950539 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:16.950822 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] PATH_INFO: `/v3/projects/ad198466643d436cab7134782f64069e` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:16.952029 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad198466643d436cab7134782f64069e)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:16.953983 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.077717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad198466643d436cab7134782f64069e', 'request_id': 'req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:17.083226 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad198466643d436cab7134782f64069e', 'request_id': 'req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:17.180277 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad198466643d436cab7134782f64069e', 'request_id': 'req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:17.191931 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] The token cache is being invalidate because project ad198466643d436cab7134782f64069e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:17.192218 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:17.192979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-d7bbb331-aeaf-4a08-9a9a-079e1f39f6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:17.194547 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1424/2869] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:16 2022] DELETE /identity/v3/projects/ad198466643d436cab7134782f64069e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:17.199390 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.199859 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.200005 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.218494 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:17.219689 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:17.249527 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:17.277694 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1446/2870] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:17.285562 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.285894 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.286160 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.304735 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:17.305980 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:17.313846 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1425/2871] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:17.318553 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.328619 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209c221-e5b6-4d23-b684-5f610df5b3d6 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.333186 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.334560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.334560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.334560 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.335157 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:53:17.335464 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.337040 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.349182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1447/2872] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:53:17.353344 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.362588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098cb31-1728-4724-bbc2-0511259aa023 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.367690 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.368326 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.368607 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.368883 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.369586 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.371451 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.461657 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1426/2873] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:17 2022] POST /identity/v3/projects => generated 359 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:17.465888 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.476563 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960506e-b715-40b4-bfb4-0af2ec559ef7 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.531139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.531826 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.532110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.532389 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.533078 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.535497 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.629807 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1448/2874] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:17 2022] POST /identity/v3/users => generated 280 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:17.634010 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.645091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99689b4-b725-4d13-9350-7041b5c5e86b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.649696 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.650423 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.650707 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.650982 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.651701 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.653315 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.664116 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1427/2875] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:17.667999 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.678091 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dd650-9723-49f3-9709-25f5beafad1c admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.682749 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.683402 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.683687 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.683966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.684678 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.686304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67d264-0e3f-47c8-9702-1d420f185804 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.697095 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1449/2876] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:17.700914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.711473 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ce58b-80db-4253-9528-6db469a14c06 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:17.716132 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:17.717003 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.717321 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.717604 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] PATH_INFO: `/v3/projects/a7bc11ec680a409fbe9ad3ce335b672b/users/5e059fbf879b4d6a9d88e66037508aac/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.731820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bc11ec680a409fbe9ad3ce335b672b, user_id=5e059fbf879b4d6a9d88e66037508aac, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:17.734495 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:17.844248 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:17.849905 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1428/2877] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:17 2022] PUT /identity/v3/projects/a7bc11ec680a409fbe9ad3ce335b672b/users/5e059fbf879b4d6a9d88e66037508aac/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:17.859863 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:17.860160 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:17.860450 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:17.887257 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] MFA Rules not processed for user `5e059fbf879b4d6a9d88e66037508aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:17.888430 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:17.956685 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1450/2878] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:17 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:17.988472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:17.990196 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:18.051688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64596e5b-bcd0-449d-8d76-8d64adb40dd0 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:18.055296 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:18.056050 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:18.056337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:18.056615 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:18.064803 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:18.066807 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80134e-e048-4f63-9c41-6a20a8f61368 service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:18.075688 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1429/2879] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:18.089552 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:18.122715 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:18.151318 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858645a-4bf9-48fd-b86e-a250c152fe55 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:18.155978 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:18.160345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:18.160345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:18.160345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] PATH_INFO: `/v3/users/5e059fbf879b4d6a9d88e66037508aac` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:18.160345 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e059fbf879b4d6a9d88e66037508aac)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:18.161016 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:18.365445 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e059fbf879b4d6a9d88e66037508aac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:18.366125 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e059fbf879b4d6a9d88e66037508aac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:18.366805 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e059fbf879b4d6a9d88e66037508aac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:18.373631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e059fbf879b4d6a9d88e66037508aac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:18.385528 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e059fbf879b4d6a9d88e66037508aac'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:18.466505 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1451/2880] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:18 2022] DELETE /identity/v3/users/5e059fbf879b4d6a9d88e66037508aac => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:18.470987 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:18.475459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:18.475459 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:18.488820 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:18.490036 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:18.547919 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1430/2881] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:18.557509 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:18.558906 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:18.616296 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a520-670f-42ae-af38-931ed54ae551 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:18.620020 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:18.627501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:18.627501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:18.627501 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:18.630666 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:18.632112 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad9c33-81e0-4d05-b226-7a9c7d207a11 service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:18.641546 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1452/2882] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:19.090826 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.092284 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:19.110647 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82a6f9-bf07-4289-8bfb-63fe1647944d None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.111468 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.112349 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.112663 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.113081 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] PATH_INFO: `/v3/projects/a7bc11ec680a409fbe9ad3ce335b672b` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.114432 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bc11ec680a409fbe9ad3ce335b672b)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.116336 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:19.249008 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bc11ec680a409fbe9ad3ce335b672b', 'request_id': 'req-2b30f1dd-a62a-450c-baec-463617282991'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:19.255337 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bc11ec680a409fbe9ad3ce335b672b', 'request_id': 'req-2b30f1dd-a62a-450c-baec-463617282991'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:19.388717 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bc11ec680a409fbe9ad3ce335b672b', 'request_id': 'req-2b30f1dd-a62a-450c-baec-463617282991'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:19.400181 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] The token cache is being invalidate because project a7bc11ec680a409fbe9ad3ce335b672b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:19.400472 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:19.401091 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-2b30f1dd-a62a-450c-baec-463617282991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:19.402650 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1431/2883] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:19 2022] DELETE /identity/v3/projects/a7bc11ec680a409fbe9ad3ce335b672b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:19.408358 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.408867 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.409003 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.427418 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:19.428631 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:19.475753 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:19.487575 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1453/2884] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:19.500291 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.500588 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.500873 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.529048 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:19.530354 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:19.538903 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1432/2885] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:19.543161 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.553977 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838568be-5d0a-4155-a090-1d7eb9ba48f7 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.559539 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80347) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.564652 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:19.586110 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1454/2886] 10.222.0.30 () {58 vars in 1152 bytes} [Tue Jul 12 14:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:53:19.590508 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.600853 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf9c28-3508-4251-b305-7e9f89a13a50 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.605858 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.606510 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.606795 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.607074 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.607796 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.610083 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:19.623920 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:19.696438 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1433/2887] 10.222.0.30 () {60 vars in 1152 bytes} [Tue Jul 12 14:53:19 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:19.700684 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.711301 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d154f-7558-43a8-b832-191364adbb34 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.766380 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.767044 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.767329 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.767602 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.768282 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.770930 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:19.940792 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1455/2888] 10.222.0.30 () {60 vars in 1143 bytes} [Tue Jul 12 14:53:19 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:53:19.945088 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.956993 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5cfe1-f8a1-4427-bba5-7d5318770f8b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.960492 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.961143 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.961456 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.961732 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.962450 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.964045 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:19.975039 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1434/2889] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:19.978809 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:19.988746 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a96fe-ab3c-41e1-943f-9d4d660e1ac3 admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:19.993304 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:19.993957 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:19.994238 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:19.994513 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:19.995264 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:19.996843 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d531fab-1662-4918-998e-76cbfe841421 admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:20.006911 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1456/2890] 10.222.0.30 () {58 vars in 1121 bytes} [Tue Jul 12 14:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:53:20.010935 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:20.021085 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7607d20-d04b-45e1-829a-f78aa0e32950 admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:20.025529 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:20.026382 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.026664 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.026942 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] PATH_INFO: `/v3/projects/c33b841ec5f84380b31a9c9b7a364710/users/e114b9e34496476c95689d0de43fe3b2/roles/0571188a7619422aac6f72fee883d8d6` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.041372 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33b841ec5f84380b31a9c9b7a364710, user_id=e114b9e34496476c95689d0de43fe3b2, role_id=0571188a7619422aac6f72fee883d8d6)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:20.043973 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:20.134915 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1435/2891] 10.222.0.30 () {60 vars in 1482 bytes} [Tue Jul 12 14:53:20 2022] PUT /identity/v3/projects/c33b841ec5f84380b31a9c9b7a364710/users/e114b9e34496476c95689d0de43fe3b2/roles/0571188a7619422aac6f72fee883d8d6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:20.145756 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] REQUEST_METHOD: `POST` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.146052 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.146336 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.172541 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.auth.core [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] MFA Rules not processed for user `e114b9e34496476c95689d0de43fe3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80347) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:20.173788 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:20.236488 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1457/2892] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:20.248038 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:20.249766 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:20.306499 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753d289-fa62-4b46-8629-24ac662eba3b admin admin] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:20.310168 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '177ef85784a94f3782aa48b60a6fb8cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:20.310812 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] REQUEST_METHOD: `GET` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.311101 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.311379 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.320193 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:20.321656 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4f877-a627-4c6f-b5d8-a7016130220f service cinder] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:20.330463 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1436/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:20.354910 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:20.403481 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:20.417410 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4c82a-2419-40c7-8095-2742cf36a922 None None] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:20.422045 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:20.422918 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.423203 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.423487 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] PATH_INFO: `/v3/users/e114b9e34496476c95689d0de43fe3b2` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.424696 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e114b9e34496476c95689d0de43fe3b2)` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:20.426659 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:20.667350 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e114b9e34496476c95689d0de43fe3b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:20.667966 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e114b9e34496476c95689d0de43fe3b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:20.668592 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e114b9e34496476c95689d0de43fe3b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:20.675758 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e114b9e34496476c95689d0de43fe3b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:20.687427 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.notifications [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e114b9e34496476c95689d0de43fe3b2'} {{(pid=80347) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:20.823723 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1458/2894] 10.222.0.30 () {58 vars in 1223 bytes} [Tue Jul 12 14:53:20 2022] DELETE /identity/v3/users/e114b9e34496476c95689d0de43fe3b2 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:20.828356 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] REQUEST_METHOD: `POST` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.828674 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.828953 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.845914 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.auth.core [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] MFA Rules not processed for user `cb516c8046b14d4c8dffdce40fe28f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80346) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:53:20.847167 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:20.905739 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1437/2895] 10.222.0.30 () {58 vars in 957 bytes} [Tue Jul 12 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:53:20.915406 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:20.916857 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:20.973387 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5468a8-8f64-4d4e-aa0d-b9e5c27f4a6b admin admin] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:20.976949 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ea4fd4eacb4664af0dc24ebfcacd42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:20.977630 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:20.977913 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:20.978189 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:20.987214 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:20.989182 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:20.998559 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1459/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:21.427333 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] Authenticating user token {{(pid=80346) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:21.428813 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.fernet_utils [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80346) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:21.443187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e033a-c949-446b-b084-bd6e000957b5 None None] Validating token access rules against request {{(pid=80346) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:21.447851 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb516c8046b14d4c8dffdce40fe28f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27c596858f1e4aabb2a1fb3cbe092011', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80346) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:21.448705 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:21.448979 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] SCRIPT_NAME: `/identity` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:21.449302 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] PATH_INFO: `/v3/projects/c33b841ec5f84380b31a9c9b7a364710` {{(pid=80346) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:21.451187 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c33b841ec5f84380b31a9c9b7a364710)` {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:21.453204 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] RBAC: Authorization granted {{(pid=80346) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:21.583184 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c33b841ec5f84380b31a9c9b7a364710', 'request_id': 'req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:21.588908 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c33b841ec5f84380b31a9c9b7a364710', 'request_id': 'req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:21.590566 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:21.714633 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c33b841ec5f84380b31a9c9b7a364710', 'request_id': 'req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e'} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:21.726135 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] The token cache is being invalidate because project c33b841ec5f84380b31a9c9b7a364710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80346) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:53:21.726418 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:21.727061 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG keystone.notifications [None req-0b3b0c2e-07bf-48c1-a9f8-b24c60f4437e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80346) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:53:21.728583 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: [pid: 80346|app: 0|req: 1438/2897] 10.222.0.30 () {58 vars in 1232 bytes} [Tue Jul 12 14:53:21 2022] DELETE /identity/v3/projects/c33b841ec5f84380b31a9c9b7a364710 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:53:30.994355 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:31.883322 dv-cinder-820027-12-iscsi devstack@keystone.service[80346]: DEBUG dbcounter [-] [80346] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=4 {{(pid=80346) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:53:35.457229 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] Authenticating user token {{(pid=80347) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:53:35.458642 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:35.526416 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4781e-7a51-4aed-97ae-9829930511ba service neutron] Validating token access rules against request {{(pid=80347) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:53:35.584151 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e40592fc25234d65a0be99d3d29869cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dfc15225d38c4588bdeedd63e0cbdd95', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80347) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:53:35.584794 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] REQUEST_METHOD: `GET` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:53:35.585076 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] SCRIPT_NAME: `/identity` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:53:35.585385 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80347) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:53:35.587187 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.fernet_utils [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80347) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:53:35.598752 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:53:35.600309 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbe1ea-c892-46c8-ab65-177c2a4c2638 service placement] RBAC: Authorization granted {{(pid=80347) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:53:35.610139 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: [pid: 80347|app: 0|req: 1460/2898] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Jul 12 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:53:45.605846 dv-cinder-820027-12-iscsi devstack@keystone.service[80347]: DEBUG dbcounter [-] [80347] Writing DB stats keystone:SELECT=58 {{(pid=80347) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}